commit 657bf290795b2eece02e63b0e876da3c64c8c5d9
parent f5d2a6e654fa9c7a569642f0c610d52533dc7cdd
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date: Fri, 9 Oct 2026 14:52:47 -0400
Merge track D (release 21 D1 attach-media, D3 playable copies + torrents, D4b the last-resort seeder) into r19/integration
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Diffstat:
49 files changed, 6749 insertions(+), 19 deletions(-)
diff --git a/COMMANDS.md b/COMMANDS.md
@@ -45,6 +45,10 @@ Run from the repo root (in the container: `docker compose exec editor pnpm archi
| `archilyzer archive-org refresh` | `<slug> [--dry-run]` | bring a channel's archive.org file records up to their provenance: a name-only mirror's title (where the item gives the file none) and upload date from its file name; offline, through the metadata history; prints old → new |
| `archilyzer wayback refresh` | `<slug> [--titles <json>] [--dry-run]` | bring a channel's Wayback Machine copies up to the Wayback rules: wayback.json (original URL, capture time), the dir renamed to its canonical id through the snapshot's reconcile (roster moved with it), and with --titles (a file of id → {title, upload\_date}) a raw file's title and date; offline, skips a record a live job holds; prints old → new |
| `archilyzer feeds backfill-metadata` | `<slug> [--feed <url>] [--dry-run]` | complete a podcast channel's records (title, date, description, duration) from its RSS feed: one fetch of the feed (default: the channel's url), no media; --dry-run counts matched / unmatched / already complete and writes nothing |
+| `archilyzer media attach` | `<slug> <source> [--items <json\|@file>] [--match <regex>] [--create-records] [--replace] [--dry-run]` | give a channel's held videos their media from a LOCAL archive (a directory, a .zip read in place, a .7z) into the saved-video store, provenance recorded (archive, entry, sha256); the id from the folder's "(<id>)" or the file's yt-dlp suffix; --create-records writes a record for a video not held; --dry-run lists each folder's class and writes nothing; offline, nothing fetched |
+| `archilyzer media playable` | `<slug> [--ids <a,b>] [--trackers <url,url>] [--root <dir>] [--dry-run]` | remux a channel's saved containers losslessly (-c copy) into browser-playable mp4 (+faststart) or webm, one single-file torrent each (no web seed; --trackers is the announce list, outside the infohash), under playable/ beside the saved-video store; keyed by the source's sha256, so a re-run is a no-op; offline |
+| `archilyzer seed` | | the home seeder of last resort: seed the playable torrents of settings.seeder.sites, each only while no other seeder has it (standby otherwise, with hysteresis), TCP + WebRTC, no DHT; runs until interrupted — behind the VPN (docker compose profile `seeder`, docker-compose.seeder.yml) |
+| `archilyzer tracker` | `[--host 127.0.0.1] [--port 8000]` | the self-hosted tracker (HTTP + WebSocket, no UDP) for the seeder's torrents only — any other infohash is refused; runs until interrupted |
| `archilyzer duplicates` | `[--threshold N] [--all-durations] [--blocking title\|duration\|both] [--near F] [--tolerance N] …` | on-demand duplicate detection (after index + stats) *(passthrough)* |
| `archilyzer posts fetch` | `--slug <channel> [--full \| --older [--floor YYYY-MM-DD] [--from YYYY-MM-DD] [--force]] [--limit N] [--pages N]` | fetch a social channel's posts into its posts corpus (--older: walk back below the oldest archived post; --pages: a forum thread's latest N pages) *(passthrough)* |
| `archilyzer posts import-html` | `<slug> <file-or-dir>… [--dry-run]` | import forum thread pages saved from a browser ("Save page as", .html) into a forum-thread channel: new posts appended, edited ones updated, nothing fetched |
@@ -82,6 +86,8 @@ Drives a running editor over HTTP (`/api/ops/*`, the same actions its pages run)
| `refresh-metadata` | refresh-metadata re-reads ONE video's metadata.info.json from its source (no subtitles, no media) on the platform's queue: {"slug", "id"}. The job's log ends with what the source now says — live\_status, formats, audio-only formats and whether any is non-fragmented, English captions, the keys that changed. An id with no data/<id>/ is refused (a refresh re-reads a video already archived), as are archive.org and Wayback records. | `pnpm ops refresh-metadata --json '{"slug":"the-quartering","id":"<videoId>"}' --wait` |
| `import-video` | — | `pnpm ops import-video --json '{"slug":"demo-archive","url":"https://archive.org/details/example-item"}'` |
| `import-archive-org` | — | `pnpm ops import-archive-org --json '{"slug":"demo-archive","item":"example-item","match":"\\.mp4$"}' --wait` |
+| `attach-media` | attach-media copies each held video's file out of a LOCAL archive into the saved-video store, as its source container (nothing fetched): {"slug", "source"} — an absolute path to a directory, a .zip (read in place) or a .7z. The id is the folder's trailing "(<id>)", else the file's yt-dlp suffix; "items": \[{"id", "path"}\] names exact files (path inside the source), "match" narrows the folders by regex. "createRecords": true writes a record for a video the channel does not hold; "replace": true re-attaches over a saved container. "dryRun": true logs each folder's class (attach, not-held, already-attached, lost, no-media, unmatched, ambiguous) and the held videos with no media in it, and writes nothing. The log ends with a summary: line; a re-run resumes. | |
+| `prepare-playable` | prepare-playable remuxes each of a channel's saved containers, losslessly (-c copy), into a browser-playable mp4 (+faststart) or webm, and makes one single-file torrent per copy, under playable/ beside the saved-video store: {"slug"}. "ids" narrows it; "trackers": \[...\] is each .torrent's announce list (none by default; the infohash does not depend on it); "root" names another playable root. A video prepared from the same source (by sha256) is skipped; "dryRun": true logs each decision. | |
| `feed-metadata` | — | `pnpm ops feed-metadata --json '{"slug":"demo-podcast","dryRun":true}' --wait` |
| `refresh-report` | — | `pnpm ops refresh-report --json '{"all":true}'` |
| `sync` | — | `pnpm ops sync --json '{"slug":"the-quartering"}' --wait` |
diff --git a/SETTINGS.md b/SETTINGS.md
@@ -48,6 +48,7 @@ A copied example PINS every default it spells — including each lane's `autoQue
| [`attribution`](#attribution) | object — see below |
| [`archiveOrg`](#archiveorg) | object — see below |
| [`publish`](#publish) | object — see below |
+| [`seeder`](#seeder) | object — see below |
## `adminTitle`
@@ -856,3 +857,33 @@ Default:
"homepage": "off"
}
```
+
+## `seeder`
+
+The home seeder of last resort (release 21): `archilyzer seed` seeds the sites' playable torrents (`archilyzer media playable`), each one only while no other seeder has it, and only behind a VPN (the docker profile `seeder`, docker-compose.seeder.yml). Not configured until `sites` names one. See SeederSettings and common/controller/seeder.ts.
+
+#### `seeder`
+
+| Key | Default | Description |
+|---|---|---|
+| `sites` | `[]` | The site ids whose playable torrents the seeder seeds: every channel of each site that has a playable manifest (`archilyzer media playable`). Empty = the seeder is not configured, and `archilyzer seed` refuses to start. |
+| `trackers` | `[]` | Announce URLs the seeder announces to and scrapes (wss:// or ws:// for browsers, http(s):// or udp:// for desktop clients). Empty = each torrent's own announce list. |
+| `maxUploadKiBps` | `0` | Upload cap across every torrent, KiB/s. 0 = unlimited. |
+| `maxConnections` | `50` | Most peer connections per torrent. Clamped to [1, 500]; default 50. |
+| `pollSeconds` | `60` | How often the seeder scrapes the trackers and decides, per torrent, whether to seed or stand by. A viewer whose last other seeder leaves mid-stream waits up to this long. Clamped to [10, 3600]; default 60. |
+| `standbyAfterSeconds` | `600` | Other seeders seen on every poll for this long → the seeder stands by on that torrent (stops announcing, closes its peers, keeps the data). The same window, with no other seeder seen, brings it back when nobody is waiting; a waiting leecher brings it back at once. Clamped to [60, 86400]; default 600. |
+| `bindInterface` | `""` | The network interface the seeder must have — the VPN tunnel's (e.g. `wg0`, `tun0`). While it is missing the seeder does not start, and stops every torrent if it disappears. Empty = not checked (the docker profile `seeder` confines the process to the tunnel either way; see docker-compose.seeder.yml). |
+
+Default:
+
+```json
+{
+ "sites": [],
+ "trackers": [],
+ "maxUploadKiBps": 0,
+ "maxConnections": 50,
+ "pollSeconds": 60,
+ "standbyAfterSeconds": 600,
+ "bindInterface": ""
+}
+```
diff --git a/common/bin/archilyzer.ts b/common/bin/archilyzer.ts
@@ -523,6 +523,82 @@ export const COMMANDS: Command[] = [
});
},
},
+ {
+ path: ["media", "attach"],
+ usage:
+ "<slug> <source> [--items <json|@file>] [--match <regex>] [--create-records] [--replace] [--dry-run] give a channel's held videos their media from a LOCAL archive (a directory, a .zip read in place, a .7z) into the saved-video store, provenance recorded (archive, entry, sha256); the id from the folder's \"(<id>)\" or the file's yt-dlp suffix; --create-records writes a record for a video not held; --dry-run lists each folder's class and writes nothing; offline, nothing fetched",
+ flags: {
+ items: "string",
+ match: "string",
+ "create-records": "boolean",
+ replace: "boolean",
+ "dry-run": "boolean",
+ },
+ maxPositionals: 2,
+ run: async ({ positionals, flags }) => {
+ const [slug, source] = positionals;
+ if (!slug || !source) {
+ console.error("media attach: which channel, from which archive? Pass <slug> <source>.");
+ return 2;
+ }
+ return (await import("./media-attach")).main({
+ slug,
+ source,
+ ...(typeof flags.items === "string" ? { items: flags.items } : {}),
+ ...(typeof flags.match === "string" ? { match: flags.match } : {}),
+ createRecords: flags["create-records"] === true,
+ replace: flags.replace === true,
+ dryRun: flags["dry-run"] === true,
+ signal: interrupted(),
+ });
+ },
+ },
+ {
+ path: ["media", "playable"],
+ usage:
+ "<slug> [--ids <a,b>] [--trackers <url,url>] [--root <dir>] [--dry-run] remux a channel's saved containers losslessly (-c copy) into browser-playable mp4 (+faststart) or webm, one single-file torrent each (no web seed; --trackers is the announce list, outside the infohash), under playable/ beside the saved-video store; keyed by the source's sha256, so a re-run is a no-op; offline",
+ flags: { ids: "string", trackers: "string", root: "string", "dry-run": "boolean" },
+ maxPositionals: 1,
+ run: async ({ positionals, flags }) => {
+ const [slug] = positionals;
+ if (!slug) {
+ console.error("media playable: which channel? Pass its slug.");
+ return 2;
+ }
+ return (await import("./media-playable")).main({
+ slug,
+ ...(typeof flags.ids === "string" ? { ids: flags.ids } : {}),
+ ...(typeof flags.trackers === "string" ? { trackers: flags.trackers } : {}),
+ ...(typeof flags.root === "string" ? { root: flags.root } : {}),
+ dryRun: flags["dry-run"] === true,
+ signal: interrupted(),
+ });
+ },
+ },
+ {
+ path: ["seed"],
+ usage:
+ "the home seeder of last resort: seed the playable torrents of settings.seeder.sites, each only while no other seeder has it (standby otherwise, with hysteresis), TCP + WebRTC, no DHT; runs until interrupted — behind the VPN (docker compose profile `seeder`, docker-compose.seeder.yml)",
+ run: async () => (await import("./seed")).seedMain({ signal: interrupted() }),
+ },
+ {
+ path: ["tracker"],
+ usage:
+ "[--host 127.0.0.1] [--port 8000] the self-hosted tracker (HTTP + WebSocket, no UDP) for the seeder's torrents only — any other infohash is refused; runs until interrupted",
+ flags: { host: "string", port: "string" },
+ run: async ({ flags }) => {
+ const port = typeof flags.port === "string" ? Number(flags.port) : 8000;
+ if (!Number.isInteger(port) || port < 0 || port > 65535) {
+ console.error("tracker: --port must be a port number");
+ return 2;
+ }
+ return (await import("./seed")).trackerMain({
+ host: typeof flags.host === "string" ? flags.host : "127.0.0.1",
+ port,
+ signal: interrupted(),
+ });
+ },
+ },
// The bins that parse their own flags, run as children with their argv
// verbatim (_spawnBin.ts says why). Each row is the whole integration.
script(["duplicates"], "duplicate-shorts.ts",
diff --git a/common/bin/doctor.test.ts b/common/bin/doctor.test.ts
@@ -971,3 +971,29 @@ test("publish: wrangler — the pinned binary or WRANGLER_BIN, run for its major
assert.deepEqual(tree(c.root), before);
assert.deepEqual(readdirSync(h), [], "nothing under HOME");
});
+
+// ── release 21 D4b: the seeder's egress ─────────────────────────────────────
+
+test("seeder: not configured is a note, and nothing is asked of the network", async () => {
+ const c = checkout();
+ let asked = 0;
+ const r = await run(c, {}, { seederEgress: async () => (asked++, { host: null, seeder: null }) });
+ assert.equal(find(r, "seeder", "seeder")?.status, "info");
+ assert.match(find(r, "seeder", "seeder")!.detail, /seeder not configured/);
+ assert.equal(asked, 0);
+ assert.equal(r.ok, true);
+});
+
+test("seeder: the same egress as the host fails; a different one is ok; an unanswered one warns", async () => {
+ const c = checkout();
+ writeFileSync(c.paths.settingsFile, JSON.stringify({ seeder: { sites: ["demo-private"] } }));
+ const same = await run(c, {}, { seederEgress: async () => ({ host: "198.51.100.7", seeder: "198.51.100.7" }) });
+ assert.equal(find(same, "seeder", "egress")?.status, "fail");
+ assert.match(find(same, "seeder", "egress")!.detail, /SAME address as this host: it is not behind the VPN/);
+ assert.equal(same.ok, false);
+ const vpn = await run(c, {}, { seederEgress: async () => ({ host: "198.51.100.7", seeder: "203.0.113.9" }) });
+ assert.equal(find(vpn, "seeder", "egress")?.status, "ok");
+ const down = await run(c, {}, { seederEgress: async () => ({ host: "198.51.100.7", seeder: null, seederError: "No such container" }) });
+ assert.equal(find(down, "seeder", "egress")?.status, "warn");
+ assert.match(find(down, "seeder", "egress")!.detail, /No such container.*profile up/);
+});
diff --git a/common/bin/doctor.ts b/common/bin/doctor.ts
@@ -18,7 +18,10 @@
// of the source repository's main. It never opens
// LMDB (the index is stat'd, not opened), never mkdirs outside the OS temp dir
// (one for `wrangler --version`'s debug log, removed after), never writes settings,
-// and never binds a port (a port is "in use" when a TCP connect succeeds). The
+// and never binds a port (a port is "in use" when a TCP connect succeeds). Its
+// one network request is the seeder's egress check (release 21) — this host's
+// address and the seeder container's, from one echo — made only when
+// `settings.seeder` names sites. The
// one process-state change is a chdir around umtool's table, which resolves a
// path from the cwd; it is put back before anything else runs.
//
@@ -99,8 +102,66 @@ export type DoctorDeps = {
// Free bytes on the filesystem holding `dir`, or null when it cannot be
// asked. Default: statfs.
freeBytes?: (dir: string) => number | null;
+ // The host's egress IP and the seeder container's, asked only when the
+ // seeder is configured. Default: probeSeederEgress.
+ seederEgress?: () => Promise<SeederEgressProbe>;
};
+// What `archilyzer doctor` learned of the seeder's way out: each egress IP, or
+// null with why it could not be asked.
+export type SeederEgressProbe = {
+ host: string | null;
+ seeder: string | null;
+ hostError?: string;
+ seederError?: string;
+};
+
+// A FAIL when the two are the same address; a warning when either could not
+// be asked (the seeder not running is not proof of a leak, nor of its absence).
+export function seederEgressVerdict(p: SeederEgressProbe): { status: CheckStatus; detail: string } {
+ if (!p.seeder) {
+ return { status: "warn", detail: `the seeder's egress could not be asked (${p.seederError ?? "no answer"}) — is the \`seeder\` profile up? (docker-compose.seeder.yml)` };
+ }
+ if (!p.host) {
+ return { status: "warn", detail: `the seeder leaves as ${p.seeder}; this host's own egress could not be asked (${p.hostError ?? "no answer"}), so it cannot be compared` };
+ }
+ if (p.host === p.seeder) {
+ return { status: "fail", detail: `the seeder leaves as ${p.seeder} — the SAME address as this host: it is not behind the VPN. Stop it.` };
+ }
+ return { status: "ok", detail: `the seeder leaves as ${p.seeder}, this host as ${p.host}: behind the VPN` };
+}
+
+// The address echo both sides ask. One plain-text GET; asked only when the
+// seeder is configured.
+const EGRESS_ECHO_URL = "https://api.ipify.org";
+// The seeder's container under docker-compose.seeder.yml (project `archilyzer`).
+const SEEDER_CONTAINER = "archilyzer-seeder-1";
+
+export async function probeSeederEgress(env: NodeJS.ProcessEnv): Promise<SeederEgressProbe> {
+ const out: SeederEgressProbe = { host: null, seeder: null };
+ const ipLike = (s: string) => (/^[0-9a-fA-F.:]{3,45}$/.test(s.trim()) ? s.trim() : null);
+ try {
+ const res = await fetch(EGRESS_ECHO_URL, { signal: AbortSignal.timeout(10_000) });
+ out.host = ipLike(await res.text());
+ if (!out.host) out.hostError = "the echo did not answer with an address";
+ } catch (err) {
+ out.hostError = (err as Error).message;
+ }
+ try {
+ const { dockerBin } = await import("../publish/build");
+ const { stdout } = await execFileP(
+ dockerBin(env),
+ ["exec", SEEDER_CONTAINER, "node", "-e", `fetch(${JSON.stringify(EGRESS_ECHO_URL)}).then((r) => r.text()).then((t) => process.stdout.write(t))`],
+ { timeout: 20_000 },
+ );
+ out.seeder = ipLike(String(stdout));
+ if (!out.seeder) out.seederError = "the echo did not answer with an address";
+ } catch (err) {
+ out.seederError = ((err as Error).message.split("\n")[0] ?? "").slice(0, 200);
+ }
+ return out;
+}
+
// Where `source publish` would read main from: the variable's path or the
// checkout's common dir (null: neither), and main's commit or why it did not
// read.
@@ -722,6 +783,24 @@ export async function collectDoctorReport(deps: DoctorDeps): Promise<DoctorRepor
? `main ${published.sourceCommit.slice(0, 12)} as ${published.mirrorHead.slice(0, 12)}, ${published.generatedAt} (${published.files} files)`
: `nothing published in ${path.join(publicDir, "source")}`);
+ // ── seeder (release 21 D4b) ──────────────────────────────────────────────
+ // The home seeder must leave through the VPN and nothing else. Its egress IP
+ // (asked from inside its container) against this host's: the same address is
+ // a FAIL — the seeder would be uploading from home. Not configured (no
+ // `settings.seeder.sites`) is a note, and asks nothing of the network.
+ const SD = "seeder";
+ {
+ const { seederConfigured } = await import("../lib/seederSettings");
+ const seeder = settings?.seeder;
+ if (!seeder || !seederConfigured(seeder)) {
+ add(SD, "seeder", "info", "seeder not configured (settings.seeder.sites is empty)");
+ } else {
+ const probe = await (deps.seederEgress ?? (() => probeSeederEgress(env)))();
+ const v = seederEgressVerdict(probe);
+ add(SD, "egress", v.status, v.detail);
+ }
+ }
+
// ── ports ────────────────────────────────────────────────────────────────
const P = "ports";
const block = await (deps.portBlock ?? (() => worktreePortBlock(root)))();
diff --git a/common/bin/media-attach.ts b/common/bin/media-attach.ts
@@ -0,0 +1,76 @@
+// `archilyzer media attach <slug> <source> [--items <json>] [--match <re>]
+// [--create-records] [--replace] [--dry-run]` — a channel's held videos given
+// their media from a local archive, offline. The work and its rules are
+// controller/attachMedia.ts; the editor runs the same function as the
+// `attach-media` job (`POST /api/ops/attach-media`).
+//
+// Nothing on the network. It copies into the saved-video store and writes
+// pointers (and, with --create-records, records) atomically; it does not see
+// the editor's queues, so not beside a job on the same channel. The channel's
+// media guard is asked here, as the job kind's `needsMedia` asks it there.
+
+import { readFile } from "node:fs/promises";
+import { getPaths } from "../lib/paths";
+import { assertChannelMediaReachable } from "../lib/channelMedia";
+import { isValidChannelSlug, readChannelConfig } from "../controller/channels";
+import { attachMedia } from "../controller/attachMedia";
+import type { AttachItem } from "../lib/attachMediaPlan";
+
+function parseItems(text: string): AttachItem[] {
+ const raw = JSON.parse(text) as unknown;
+ if (!Array.isArray(raw) || raw.length === 0) throw new Error("--items: a non-empty JSON array of {id, path}");
+ return raw.map((e, i) => {
+ const r = e as Record<string, unknown>;
+ if (typeof r?.id !== "string" || !r.id || /[/\\\0]/.test(r.id) || typeof r.path !== "string" || !r.path) {
+ throw new Error(`--items[${i}]: an object {"id": "<video id>", "path": "<entry in the source>"}`);
+ }
+ return { id: r.id, path: r.path };
+ });
+}
+
+export async function main(opts: {
+ slug: string;
+ source: string;
+ items?: string;
+ match?: string;
+ createRecords: boolean;
+ replace: boolean;
+ dryRun: boolean;
+ signal?: AbortSignal;
+}): Promise<number> {
+ if (!isValidChannelSlug(opts.slug)) {
+ console.error(`media attach: "${opts.slug}" is not a channel slug`);
+ return 2;
+ }
+ try {
+ const paths = getPaths();
+ const channelConfig = await readChannelConfig(paths, opts.slug);
+ if (!channelConfig) {
+ console.error(`media attach: no channel "${opts.slug}"`);
+ return 2;
+ }
+ if (!opts.dryRun) await assertChannelMediaReachable(paths, opts.slug, channelConfig);
+ // --items is a JSON array, inline or "@file".
+ const items =
+ opts.items === undefined
+ ? undefined
+ : parseItems(opts.items.startsWith("@") ? await readFile(opts.items.slice(1), "utf8") : opts.items);
+ const result = await attachMedia({
+ paths,
+ slug: opts.slug,
+ channelConfig,
+ source: opts.source,
+ ...(items ? { items } : {}),
+ ...(opts.match !== undefined ? { match: opts.match } : {}),
+ ...(opts.createRecords ? { createRecords: true } : {}),
+ ...(opts.replace ? { replace: true } : {}),
+ ...(opts.dryRun ? { dryRun: true } : {}),
+ onLog: (line) => console.log(line.replace(/\n$/, "")),
+ ...(opts.signal ? { signal: opts.signal } : {}),
+ });
+ return result.failed.length > 0 || (result.stopped && result.stopped !== "drained") ? 1 : 0;
+ } catch (err) {
+ console.error(`media attach: ${(err as Error).message}`);
+ return 1;
+ }
+}
diff --git a/common/bin/media-playable.ts b/common/bin/media-playable.ts
@@ -0,0 +1,61 @@
+// `archilyzer media playable <slug> [--ids a,b] [--trackers u1,u2] [--root
+// <dir>] [--dry-run]` — a channel's saved containers remuxed losslessly into
+// browser-playable copies, with one torrent each, offline. The work and its
+// rules are controller/preparePlayable.ts; the editor runs the same function
+// as the `prepare-playable` job (`POST /api/ops/prepare-playable`).
+//
+// Disk-bound and long (a remux reads and writes every byte): run it where the
+// editor's own prepare-playable is not running.
+
+import path from "node:path";
+import { getPaths } from "../lib/paths";
+import { assertChannelMediaReachable } from "../lib/channelMedia";
+import { isValidChannelSlug, readChannelConfig } from "../controller/channels";
+import { preparePlayable } from "../controller/preparePlayable";
+
+const list = (s: string | undefined) =>
+ s === undefined ? undefined : s.split(",").map((x) => x.trim()).filter(Boolean);
+
+export async function main(opts: {
+ slug: string;
+ ids?: string;
+ trackers?: string;
+ root?: string;
+ dryRun: boolean;
+ signal?: AbortSignal;
+}): Promise<number> {
+ if (!isValidChannelSlug(opts.slug)) {
+ console.error(`media playable: "${opts.slug}" is not a channel slug`);
+ return 2;
+ }
+ if (opts.root !== undefined && !path.isAbsolute(opts.root)) {
+ console.error("media playable: --root must be an absolute path");
+ return 2;
+ }
+ try {
+ const paths = getPaths();
+ const channelConfig = await readChannelConfig(paths, opts.slug);
+ if (!channelConfig) {
+ console.error(`media playable: no channel "${opts.slug}"`);
+ return 2;
+ }
+ if (!opts.dryRun) await assertChannelMediaReachable(paths, opts.slug, channelConfig);
+ const ids = list(opts.ids);
+ const trackers = list(opts.trackers);
+ const result = await preparePlayable({
+ paths,
+ slug: opts.slug,
+ channelConfig,
+ ...(opts.root ? { root: opts.root } : {}),
+ ...(ids ? { ids } : {}),
+ ...(trackers ? { trackers } : {}),
+ ...(opts.dryRun ? { dryRun: true } : {}),
+ onLog: (line) => console.log(line.replace(/\n$/, "")),
+ ...(opts.signal ? { signal: opts.signal } : {}),
+ });
+ return result.failed.length > 0 || (result.stopped && result.stopped !== "drained") ? 1 : 0;
+ } catch (err) {
+ console.error(`media playable: ${(err as Error).message}`);
+ return 1;
+ }
+}
diff --git a/common/bin/seed.ts b/common/bin/seed.ts
@@ -0,0 +1,65 @@
+// `archilyzer seed` and `archilyzer tracker` (release 21 D4b). The work and
+// its rules are controller/seeder.ts; the settings are `settings.seeder`
+// (SETTINGS.md). Both run until interrupted.
+//
+// RUN THEM BEHIND THE VPN. The docker profile `seeder`
+// (docker-compose.seeder.yml) is the supported way: its namespace has no route
+// but the tunnel. Run on a bare host, the seeder uploads from the home address.
+
+import { getPaths } from "../lib/paths";
+import { getSettings } from "../lib/settings";
+import { seederConfigured } from "../lib/seederSettings";
+import { listSeedTargets, runSeeder, startTracker } from "../controller/seeder";
+
+export async function seedMain(opts: { signal: AbortSignal }): Promise<number> {
+ const settings = getSettings().seeder;
+ if (!seederConfigured(settings)) {
+ console.error("seed: the seeder is not configured — name the sites it seeds in settings.seeder.sites (SETTINGS.md)");
+ return 2;
+ }
+ try {
+ await runSeeder({
+ paths: getPaths(),
+ settings,
+ onLog: (line) => console.log(line.replace(/\n$/, "")),
+ signal: opts.signal,
+ });
+ return 0;
+ } catch (err) {
+ console.error(`seed: ${(err as Error).message}`);
+ return 1;
+ }
+}
+
+// How long the tracker trusts its list of infohashes before re-reading the
+// manifests (an announce arrives every few minutes per peer; a manifest read
+// per announce would be a disk walk per announce).
+const ALLOWED_TTL_MS = 60_000;
+
+export async function trackerMain(opts: { host: string; port: number; signal: AbortSignal }): Promise<number> {
+ const settings = getSettings().seeder;
+ if (!seederConfigured(settings)) {
+ console.error("tracker: it tracks the seeder's torrents, and the seeder is not configured — settings.seeder.sites is empty");
+ return 2;
+ }
+ const paths = getPaths();
+ let cache: { at: number; set: Set<string> } | null = null;
+ const allowed = async () => {
+ if (!cache || Date.now() - cache.at > ALLOWED_TTL_MS) {
+ const targets = await listSeedTargets(paths, settings, (l) => console.log(l));
+ cache = { at: Date.now(), set: new Set(targets.map((t) => t.infoHash)) };
+ }
+ return cache.set;
+ };
+ try {
+ const n = (await allowed()).size;
+ const tracker = await startTracker({ host: opts.host, port: opts.port, allowed, onLog: (l) => console.log(l) });
+ console.log(`tracker: ${n} torrent(s) tracked; nothing else is`);
+ await new Promise<void>((resolve) => opts.signal.addEventListener("abort", () => resolve(), { once: true }));
+ await tracker.close();
+ return 0;
+ } catch (err) {
+ console.error(`tracker: ${(err as Error).message}`);
+ return 1;
+ }
+}
diff --git a/common/controller/attachMedia.test.ts b/common/controller/attachMedia.test.ts
@@ -0,0 +1,297 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import { createHash } from "node:crypto";
+import { mkdir, mkdtemp, readFile, readdir, rm, stat, writeFile } from "node:fs/promises";
+import { tmpdir } from "node:os";
+import path from "node:path";
+import { execa } from "execa";
+import type { Paths } from "../lib/paths";
+import type { ChannelConfig } from "../lib/channelConfig";
+import { makeZip, type ZipFixtureEntry } from "../testing/makeZip";
+import { loadSavedVideo } from "../lib/savedVideo-server";
+import { savedVideoPath } from "../lib/savedVideo";
+import { SAVED_VIDEOS_MARKER_FILENAME } from "../lib/savedVideoStore";
+import { attachMedia, parse7zListing, type AttachMediaOptions } from "./attachMedia";
+
+// Run with: pnpm --filter yt-dlp-transcript-common exec tsx --test controller/attachMedia.test.ts
+//
+// A synthetic archive in the pilot's shape (release 21 D1): yt-dlp folders,
+// bare-id folders with description.txt, a [LOST] folder. Every byte is made
+// here; nothing reads a real archive or corpus.
+
+const SLUG = "demo";
+const sha = (b: Buffer) => createHash("sha256").update(b).digest("hex");
+const bytes = (n: number, seed: number) => Buffer.from(Array.from({ length: n }, (_, i) => (i * seed + 3) & 0xff));
+
+const A = bytes(50_000, 7); // held, yt-dlp folder, stored
+const C = bytes(40_000, 11); // not held, bare id + description.txt, deflated
+const D = bytes(30_000, 13); // not held, yt-dlp folder with info.json
+const INFO_D = JSON.stringify({ id: "ddddddddddd", title: "From yt-dlp", extractor_key: "Youtube", upload_date: "20190101" });
+
+const ENTRIES: ZipFixtureEntry[] = [
+ { name: "YouTube/" },
+ { name: "YouTube/001 - Intro - (aaaaaaaaaaa)/Intro-aaaaaaaaaaa.mkv", data: A },
+ { name: "YouTube/001 - Intro - (aaaaaaaaaaa)/Intro-aaaaaaaaaaa.info.json", data: '{"id":"aaaaaaaaaaa"}' },
+ { name: "YouTube/002 [LOST] - Gone - (bbbbbbbbbbb)/description.txt", data: "lost" },
+ { name: "YouTube/003 - Episode 2 - (ccccccccccc)/ccccccccccc.mp4", data: C, deflate: true },
+ { name: "YouTube/003 - Episode 2 - (ccccccccccc)/description.txt", data: "Published on July 13th 2018\n\nTips" },
+ { name: "YouTube/003 - Episode 2 - (ccccccccccc)/source.txt", data: "Video source: somewhere" },
+ { name: "YouTube/004 - Other - (ddddddddddd)/Other-ddddddddddd.webm", data: D },
+ { name: "YouTube/004 - Other - (ddddddddddd)/Other-ddddddddddd.info.json", data: INFO_D },
+];
+
+type Ctx = { dir: string; paths: Paths; zip: string; lines: string[]; opts: (o?: Partial<AttachMediaOptions>) => AttachMediaOptions };
+
+async function withFixture(fn: (ctx: Ctx) => Promise<void>, entries = ENTRIES): Promise<void> {
+ const dir = await mkdtemp(path.join(tmpdir(), "ttb-attach-"));
+ const transcriptsDir = path.join(dir, "corpus");
+ const paths = {
+ transcriptsDir,
+ channelsDir: path.join(transcriptsDir, "channels"),
+ savedVideosDir: path.join(dir, "store"),
+ ffprobeBin: "ffprobe-not-used",
+ } as Paths;
+ await mkdir(paths.savedVideosDir, { recursive: true });
+ // Held: aaaaaaaaaaa (with media in the archive), bbbbbbbbbbb (LOST), hhhhhhhhhhh (no folder).
+ for (const id of ["aaaaaaaaaaa", "bbbbbbbbbbb", "hhhhhhhhhhh"]) {
+ const vd = path.join(paths.channelsDir, SLUG, "data", id);
+ await mkdir(vd, { recursive: true });
+ await writeFile(path.join(vd, "metadata.info.json"), JSON.stringify({ id, title: id }));
+ }
+ const zip = path.join(dir, "archive", "YouTube.zip");
+ await mkdir(path.dirname(zip), { recursive: true });
+ await writeFile(zip, makeZip(entries));
+ const lines: string[] = [];
+ const opts = (o: Partial<AttachMediaOptions> = {}): AttachMediaOptions => ({
+ paths,
+ slug: SLUG,
+ channelConfig: { name: "Demo", handling: "youtube", platform: "youtube" } as unknown as ChannelConfig,
+ source: zip,
+ onLog: (l) => lines.push(l),
+ deps: { probeDuration: async () => 61.5, now: () => new Date("2026-10-09T12:00:00Z") },
+ ...o,
+ });
+ try {
+ await fn({ dir, paths, zip, lines, opts });
+ } finally {
+ await rm(dir, { recursive: true, force: true });
+ }
+}
+
+const videoDir = (paths: Paths, id: string) => path.join(paths.channelsDir, SLUG, "data", id);
+
+test("dryRun lists every class and writes nothing", async () => {
+ await withFixture(async ({ paths, zip, opts, lines }) => {
+ const zipBefore = sha(await readFile(zip));
+ const r = await attachMedia(opts({ dryRun: true }));
+ assert.equal(r.counts.attach, 1);
+ assert.equal(r.counts["not-held"], 2);
+ assert.equal(r.counts.lost, 1);
+ assert.deepEqual(r.notHeld.sort(), ["ccccccccccc", "ddddddddddd"]);
+ assert.deepEqual(r.heldWithoutMedia, ["bbbbbbbbbbb", "hhhhhhhhhhh"]);
+ assert.deepEqual(r.attached, []);
+ assert.equal(await loadSavedVideo(videoDir(paths, "aaaaaaaaaaa")), null);
+ assert.deepEqual(await readdir(paths.savedVideosDir), []);
+ assert.equal(sha(await readFile(zip)), zipBefore, "the archive is never written");
+ assert.ok(lines.some((l) => l.startsWith("summary: ")));
+ });
+});
+
+test("attaches a held video through the store, with local-archive provenance", async () => {
+ await withFixture(async ({ paths, zip, opts }) => {
+ const zipBefore = sha(await readFile(zip));
+ const r = await attachMedia(opts());
+ assert.deepEqual(r.attached, ["aaaaaaaaaaa"]);
+ assert.deepEqual(r.failed, []);
+ const pointer = await loadSavedVideo(videoDir(paths, "aaaaaaaaaaa"));
+ assert.ok(pointer);
+ assert.equal(pointer.file, "source-media.mkv");
+ assert.equal(pointer.dir, path.join(paths.savedVideosDir, SLUG, "aaaaaaaaaaa"));
+ assert.equal(pointer.keepReason, "pin");
+ assert.equal(pointer.bytes, A.length);
+ assert.equal(pointer.sha256, sha(A));
+ assert.deepEqual(pointer.origin, {
+ requestedBy: "attach-media",
+ kind: "local-archive",
+ archive: zip,
+ entry: "YouTube/001 - Intro - (aaaaaaaaaaa)/Intro-aaaaaaaaaaa.mkv",
+ sha256: sha(A),
+ attachedAt: "2026-10-09T12:00:00.000Z",
+ });
+ assert.deepEqual(await readFile(savedVideoPath(pointer)), A);
+ // No staging file left beside the store dir; nothing copied into data/.
+ assert.deepEqual(await readdir(path.join(paths.savedVideosDir, SLUG)), ["aaaaaaaaaaa"]);
+ assert.deepEqual((await readdir(videoDir(paths, "aaaaaaaaaaa"))).sort(), ["metadata.info.json", "saved-video.json"]);
+ assert.equal(sha(await readFile(zip)), zipBefore);
+ // Not held, no createRecords: listed, nothing written for them.
+ await assert.rejects(stat(videoDir(paths, "ccccccccccc")));
+ });
+});
+
+test("a re-run is a no-op (already attached), and refuses over a pointer unless replace", async () => {
+ await withFixture(async ({ paths, opts }) => {
+ await attachMedia(opts());
+ const first = await loadSavedVideo(videoDir(paths, "aaaaaaaaaaa"));
+ const again = await attachMedia(opts());
+ assert.deepEqual(again.attached, []);
+ assert.deepEqual(again.alreadyAttached, ["aaaaaaaaaaa"]);
+ assert.deepEqual(await loadSavedVideo(videoDir(paths, "aaaaaaaaaaa")), first);
+
+ const replaced = await attachMedia(opts({ replace: true, deps: { now: () => new Date("2026-10-10T00:00:00Z") } }));
+ assert.deepEqual(replaced.attached, ["aaaaaaaaaaa"]);
+ const after = await loadSavedVideo(videoDir(paths, "aaaaaaaaaaa"));
+ assert.equal(after?.origin?.attachedAt, "2026-10-10T00:00:00.000Z");
+ assert.deepEqual(await readFile(savedVideoPath(after!)), A);
+ });
+});
+
+test("replace removes a previous container of another name", async () => {
+ await withFixture(async ({ paths, opts }) => {
+ // A container persisted earlier as .mp4 (a download, say).
+ const vd = videoDir(paths, "aaaaaaaaaaa");
+ const storeDir = path.join(paths.savedVideosDir, SLUG, "aaaaaaaaaaa");
+ await mkdir(storeDir, { recursive: true });
+ await writeFile(path.join(storeDir, "source-media.mp4"), "old");
+ await writeFile(
+ path.join(vd, "saved-video.json"),
+ JSON.stringify({ storedAt: "x", dir: storeDir, file: "source-media.mp4", bytes: 3, keepReason: "override" }),
+ );
+ const refused = await attachMedia(opts());
+ assert.deepEqual(refused.alreadyAttached, ["aaaaaaaaaaa"]);
+ await attachMedia(opts({ replace: true }));
+ assert.deepEqual((await readdir(storeDir)).sort(), ["source-media.mkv"]);
+ });
+});
+
+test("createRecords: a record from the folder's .info.json, or from description.txt and its name", async () => {
+ await withFixture(async ({ paths, opts }) => {
+ const r = await attachMedia(opts({ createRecords: true }));
+ assert.deepEqual(r.created.sort(), ["ccccccccccc", "ddddddddddd"]);
+ assert.deepEqual(r.attached.sort(), ["aaaaaaaaaaa", "ccccccccccc", "ddddddddddd"]);
+
+ const d = await readFile(path.join(videoDir(paths, "ddddddddddd"), "metadata.info.json"), "utf8");
+ assert.deepEqual(JSON.parse(d), JSON.parse(INFO_D));
+
+ const c = JSON.parse(await readFile(path.join(videoDir(paths, "ccccccccccc"), "metadata.info.json"), "utf8"));
+ assert.equal(c.title, "Episode 2");
+ assert.equal(c.upload_date, "20180713");
+ assert.equal(c.duration, 61.5);
+ assert.equal(c.webpage_url, "https://www.youtube.com/watch?v=ccccccccccc");
+
+ // The deflated entry came out whole.
+ const pc = await loadSavedVideo(videoDir(paths, "ccccccccccc"));
+ assert.deepEqual(await readFile(savedVideoPath(pc!)), C);
+ assert.equal(pc?.file, "source-media.mp4");
+
+ // The channel's archive file gained a line per new record, as a download's would.
+ const archive = await readFile(path.join(paths.channelsDir, SLUG, "archive"), "utf8");
+ assert.deepEqual(archive.trim().split("\n").sort(), ["youtube ccccccccccc", "youtube ddddddddddd"]);
+ });
+});
+
+test("a .info.json for another id is refused, and nothing is left behind", async () => {
+ const entries = ENTRIES.map((e) =>
+ e.name.endsWith("Other-ddddddddddd.info.json") ? { ...e, data: '{"id":"zzzzzzzzzzz"}' } : e,
+ );
+ await withFixture(async ({ paths, opts }) => {
+ const r = await attachMedia(opts({ createRecords: true }));
+ assert.deepEqual(r.failed.map((x) => x.id), ["ddddddddddd"]);
+ assert.match(r.failed[0].error, /not ddddddddddd/);
+ await assert.rejects(stat(videoDir(paths, "ddddddddddd")));
+ assert.deepEqual((await readdir(path.join(paths.savedVideosDir, SLUG))).sort(), ["aaaaaaaaaaa", "ccccccccccc"]);
+ }, entries);
+});
+
+test("an explicit item attaches a named file to a named id", async () => {
+ await withFixture(async ({ paths, opts }) => {
+ const r = await attachMedia(
+ opts({ items: [{ id: "hhhhhhhhhhh", path: "YouTube/004 - Other - (ddddddddddd)/Other-ddddddddddd.webm" }] }),
+ );
+ assert.deepEqual(r.attached, ["hhhhhhhhhhh"]);
+ const p = await loadSavedVideo(videoDir(paths, "hhhhhhhhhhh"));
+ assert.deepEqual(await readFile(savedVideoPath(p!)), D);
+ assert.equal(p?.origin?.entry, "YouTube/004 - Other - (ddddddddddd)/Other-ddddddddddd.webm");
+ });
+});
+
+test("a directory source attaches the same way", async () => {
+ await withFixture(async ({ dir, paths, opts }) => {
+ const src = path.join(dir, "unpacked");
+ const folder = path.join(src, "001 - Intro - (aaaaaaaaaaa)");
+ await mkdir(folder, { recursive: true });
+ await writeFile(path.join(folder, "Intro-aaaaaaaaaaa.mp4"), A);
+ const r = await attachMedia(opts({ source: src }));
+ assert.deepEqual(r.attached, ["aaaaaaaaaaa"]);
+ const p = await loadSavedVideo(videoDir(paths, "aaaaaaaaaaa"));
+ assert.equal(p?.origin?.archive, src);
+ assert.equal(p?.origin?.entry, "001 - Intro - (aaaaaaaaaaa)/Intro-aaaaaaaaaaa.mp4");
+ // The directory source is read, never moved.
+ assert.deepEqual(await readFile(path.join(folder, "Intro-aaaaaaaaaaa.mp4")), A);
+ });
+});
+
+test("guards: a store being moved stops the run; a store root that is not there refuses", async () => {
+ await withFixture(async ({ paths, opts }) => {
+ // The in-corpus store path is what the marker guards; point the store there.
+ const inCorpus = { ...paths, savedVideosDir: path.join(paths.transcriptsDir, "saved-videos") } as Paths;
+ await mkdir(inCorpus.savedVideosDir, { recursive: true });
+ await writeFile(
+ path.join(paths.transcriptsDir, SAVED_VIDEOS_MARKER_FILENAME),
+ JSON.stringify({ target: "/elsewhere", direction: "out", startedAt: "2026-10-09T00:00:00Z", phase: "copy" }),
+ );
+ const held = await attachMedia(opts({ paths: inCorpus }));
+ assert.deepEqual(held.attached, []);
+ assert.ok(held.stopped);
+
+ const gone = { ...paths, savedVideosDir: path.join(paths.transcriptsDir, "no-such-store") } as Paths;
+ await assert.rejects(attachMedia(opts({ paths: gone })), /is not there/);
+ await assert.rejects(stat(gone.savedVideosDir), "an unmounted store is never materialised");
+ });
+});
+
+test("drain stops before the next video", async () => {
+ await withFixture(async ({ opts }) => {
+ const drain = new AbortController();
+ drain.abort();
+ const r = await attachMedia(opts({ createRecords: true, drainSignal: drain.signal }));
+ assert.equal(r.stopped, "drained");
+ assert.deepEqual(r.attached, []);
+ });
+});
+
+test("7z listing parse, and a .7z source when 7z is installed", async (t) => {
+ const listing = [
+ "Path = /x/a.7z",
+ "Type = 7z",
+ "",
+ "----------",
+ "Path = dir",
+ "Folder = +",
+ "Size = 0",
+ "",
+ "Path = dir/001 - X - (aaaaaaaaaaa)/x.mp4",
+ "Folder = -",
+ "Size = 50000",
+ "Attributes = A",
+ "",
+ ].join("\n");
+ assert.deepEqual(parse7zListing(listing), [{ path: "dir/001 - X - (aaaaaaaaaaa)/x.mp4", size: 50000 }]);
+
+ const probe = await execa("7z", ["i"], { reject: false }).catch(() => null);
+ if (!probe || probe.exitCode !== 0) {
+ t.skip("7z is not installed");
+ return;
+ }
+ await withFixture(async ({ dir, paths, opts }) => {
+ const src = path.join(dir, "pack");
+ await mkdir(path.join(src, "001 - Intro [x] - (aaaaaaaaaaa)"), { recursive: true });
+ await writeFile(path.join(src, "001 - Intro [x] - (aaaaaaaaaaa)", "Intro-aaaaaaaaaaa.mp4"), A);
+ const archive = path.join(dir, "pack.7z");
+ await execa("7z", ["a", "-bd", archive, "."], { cwd: src });
+ const r = await attachMedia(opts({ source: archive }));
+ assert.deepEqual(r.attached, ["aaaaaaaaaaa"]);
+ const p = await loadSavedVideo(videoDir(paths, "aaaaaaaaaaa"));
+ assert.deepEqual(await readFile(savedVideoPath(p!)), A);
+ assert.equal(p?.sha256, sha(A));
+ });
+});
diff --git a/common/controller/attachMedia.ts b/common/controller/attachMedia.ts
@@ -0,0 +1,503 @@
+// ATTACH LOCAL MEDIA TO HELD VIDEOS (release 21 D1, the `attach-media` job).
+//
+// A channel whose platform copies are gone (a deleted YouTube channel) can
+// still be clipped when somebody archived its videos: this takes such an
+// archive — a directory, a .zip read IN PLACE, or a .7z — finds the video file
+// for each held record (lib/attachMediaPlan.ts says how an id is read), and
+// puts it in the saved-video store as that video's source container, so a
+// clip window, a report's evidence cut and report-to-video all cut it
+// locally. Nothing is fetched.
+//
+// THE ONE WRITER. The container lands through `persistSourceVideo`, as every
+// persisted source does: `source-media.<ext>` in the store, a pointer in
+// `data/<id>/saved-video.json`, `keepReason: "pin"` (never pruned), and an
+// origin `{kind: "local-archive", archive, entry, sha256, attachedAt}` — which
+// archive, which entry, and the hash of the bytes. The bytes are copied into a
+// staging file on the STORE's disk first (a stored zip entry is a byte range
+// of the archive; a deflated one is inflated; a 7z entry is extracted by `7z`)
+// and then renamed into place, so 36 GB never passes through the corpus SSD.
+//
+// NEVER WRITES TO THE ARCHIVE. Every read of it is read-only.
+//
+// REFUSES OVER AN EXISTING POINTER unless `replace` — a container already in
+// the store, from a download or an earlier attach, is somebody's decision. A
+// replaced container that had another name is removed once the new pointer is
+// written (a same-named one was replaced by the rename).
+//
+// NOT HELD. A video in the archive the channel has no record of is listed;
+// with `createRecords` a record is written first — `metadata.info.json` from
+// the folder's own yt-dlp `.info.json`, else built from its description.txt
+// and name (lib/attachMediaPlan.ts `archiveFolderInfoJson`) — through the
+// metadata history, and the channel's `archive` file gains its line, as a
+// download would. Transcribing it is a separate step.
+//
+// GUARDED: the job kind declares `needsMedia` (runManagedFunction asks the
+// channel's media guard), every store write asks
+// `assertSavedVideosStoreWritable`, the store's root must exist (an unmounted
+// drive is never materialised by a mkdir), and each copy needs its size plus a
+// margin free on the store's disk — short of it the run stops, and a re-run
+// resumes (an attached video is "already attached").
+//
+// DRAIN AND CANCEL are honoured between videos; a cancel also aborts the copy
+// in flight and removes its staging file.
+
+import path from "node:path";
+import { appendFile, mkdir, readFile, readdir, rm, stat } from "node:fs/promises";
+import { createWriteStream } from "node:fs";
+import { pipeline } from "node:stream/promises";
+import { Transform, type TransformCallback } from "node:stream";
+import { createHash } from "node:crypto";
+import { execa } from "execa";
+import type { Paths } from "../lib/paths";
+import type { ChannelConfig } from "../lib/channelConfig";
+import {
+ savedVideoDir,
+ savedVideoPath,
+ savedVideoRoot,
+ type SavedVideoOrigin,
+} from "../lib/savedVideo";
+import { loadSavedVideo, persistSourceVideo } from "../lib/savedVideo-server";
+import { assertSavedVideosStoreWritable } from "../lib/savedVideoStore";
+import { writeJsonAtomic } from "../lib/jsonFile-server";
+import { withMetadataHistory } from "../lib/metadataHistory-server";
+import { getFreeBytes } from "../lib/diskSpace";
+import { formatBytes } from "../lib/format";
+import { probeMediaDurationSec } from "../ytdlp/ffprobeDuration";
+import {
+ archiveFolderInfoJson,
+ countAttachRows,
+ groupArchiveFolders,
+ planAttach,
+ type AttachItem,
+ type AttachPlan,
+ type AttachRow,
+ type AttachRowClass,
+ type SourceFile,
+} from "../lib/attachMediaPlan";
+import {
+ copyFileHashed,
+ copyZipEntry,
+ readZipEntries,
+ readZipEntryText,
+ type CopiedEntry,
+ type ZipEntry,
+} from "../lib/zipReader";
+
+export const ATTACH_MEDIA_REQUESTER = "attach-media";
+
+// Free space the store's disk must keep beyond each copy.
+export const ATTACH_MEDIA_DISK_MARGIN_BYTES = 2 * 1024 ** 3;
+
+// --- sources ----------------------------------------------------------------
+
+export type ArchiveSource = {
+ kind: "zip" | "7z" | "dir";
+ // The archive's absolute path (what the origin records).
+ archive: string;
+ files: SourceFile[];
+ readText(file: SourceFile): Promise<string>;
+ copyOut(file: SourceFile, dest: string, signal?: AbortSignal): Promise<CopiedEntry>;
+};
+
+async function openZipSource(archive: string): Promise<ArchiveSource> {
+ const entries = await readZipEntries(archive);
+ const byName = new Map<string, ZipEntry>();
+ for (const e of entries) if (!e.isDirectory) byName.set(e.name, e);
+ const entry = (f: SourceFile) => {
+ const e = byName.get(f.path);
+ if (!e) throw new Error(`${f.path} is not in ${archive}`);
+ return e;
+ };
+ return {
+ kind: "zip",
+ archive,
+ files: [...byName.values()].map((e) => ({ path: e.name, size: e.size })),
+ readText: (f) => readZipEntryText(archive, entry(f)),
+ copyOut: (f, dest, signal) => copyZipEntry(archive, entry(f), dest, signal ? { signal } : {}),
+ };
+}
+
+async function walkDir(root: string, rel = ""): Promise<SourceFile[]> {
+ const out: SourceFile[] = [];
+ const entries = await readdir(path.join(root, rel), { withFileTypes: true });
+ for (const d of entries.sort((a, b) => a.name.localeCompare(b.name))) {
+ const p = rel ? `${rel}/${d.name}` : d.name;
+ if (d.isDirectory()) out.push(...(await walkDir(root, p)));
+ else if (d.isFile() || d.isSymbolicLink()) {
+ const st = await stat(path.join(root, p)).catch(() => null);
+ if (st?.isFile()) out.push({ path: p, size: st.size });
+ }
+ }
+ return out;
+}
+
+async function openDirSource(dir: string): Promise<ArchiveSource> {
+ const files = await walkDir(dir);
+ const abs = (f: SourceFile) => path.join(dir, ...f.path.split("/"));
+ return {
+ kind: "dir",
+ archive: dir,
+ files,
+ readText: async (f) => {
+ if (f.size > 4 * 1024 * 1024) throw new Error(`${f.path} is too large to read as text`);
+ return readFile(abs(f), "utf8");
+ },
+ copyOut: (f, dest, signal) => copyFileHashed(abs(f), dest, signal ? { signal } : {}),
+ };
+}
+
+class HashCount extends Transform {
+ bytes = 0;
+ readonly hash = createHash("sha256");
+ _transform(chunk: Buffer, _e: BufferEncoding, cb: TransformCallback): void {
+ this.bytes += chunk.length;
+ this.hash.update(chunk);
+ cb(null, chunk);
+ }
+}
+
+// `7z l -slt` output → files. Blocks are separated by blank lines; the
+// archive's own block (before "----------") is skipped.
+export function parse7zListing(text: string): SourceFile[] {
+ const body = text.split(/^-{10,}\s*$/m).slice(1).join("\n");
+ const out: SourceFile[] = [];
+ for (const block of body.split(/\r?\n\s*\r?\n/)) {
+ const field = (k: string) => new RegExp(`^${k} = (.*)$`, "m").exec(block)?.[1];
+ const p = field("Path");
+ if (!p) continue;
+ const folder = field("Folder");
+ const attrs = field("Attributes") ?? "";
+ if (folder === "+" || attrs.startsWith("D")) continue;
+ out.push({ path: p.replace(/\\/g, "/"), size: Number(field("Size") ?? 0) || 0 });
+ }
+ return out;
+}
+
+async function open7zSource(archive: string, sevenZipBin: string): Promise<ArchiveSource> {
+ const listed = await execa(sevenZipBin, ["l", "-slt", archive], { reject: false });
+ if (listed.exitCode !== 0) {
+ throw new Error(`${sevenZipBin} could not list ${archive}: ${String(listed.stderr).trim() || `exit ${listed.exitCode}`}`);
+ }
+ const files = parse7zListing(String(listed.stdout));
+ // `-spd`: the entry name is a name, not a wildcard pattern.
+ const extract = (f: SourceFile, signal?: AbortSignal) =>
+ execa(sevenZipBin, ["e", "-so", "-spd", archive, f.path], {
+ buffer: false,
+ stdout: "pipe",
+ ...(signal ? { cancelSignal: signal } : {}),
+ });
+ return {
+ kind: "7z",
+ archive,
+ files,
+ readText: async (f) => {
+ if (f.size > 4 * 1024 * 1024) throw new Error(`${f.path} is too large to read as text`);
+ const r = await execa(sevenZipBin, ["e", "-so", "-spd", archive, f.path], { encoding: "utf8" });
+ return String(r.stdout);
+ },
+ copyOut: async (f, dest, signal) => {
+ const child = extract(f, signal);
+ const hc = new HashCount();
+ await Promise.all([
+ pipeline(child.stdout!, hc, createWriteStream(dest, { flags: "wx" })),
+ child,
+ ]);
+ if (hc.bytes !== f.size) {
+ throw new Error(`${f.path}: ${hc.bytes} bytes extracted, the archive says ${f.size}`);
+ }
+ return { bytes: hc.bytes, sha256: hc.hash.digest("hex") };
+ },
+ };
+}
+
+export async function openArchiveSource(
+ source: string,
+ opts: { sevenZipBin?: string } = {},
+): Promise<ArchiveSource> {
+ if (!path.isAbsolute(source)) {
+ throw new Error(`"${source}" is not an absolute path`);
+ }
+ const st = await stat(source).catch(() => null);
+ if (!st) throw new Error(`${source} does not exist (is its drive mounted?)`);
+ if (st.isDirectory()) return openDirSource(source);
+ const lower = source.toLowerCase();
+ if (lower.endsWith(".zip")) return openZipSource(source);
+ if (lower.endsWith(".7z")) return open7zSource(source, opts.sevenZipBin ?? "7z");
+ throw new Error(`${source}: a source is a directory, a .zip or a .7z`);
+}
+
+// --- the run ----------------------------------------------------------------
+
+export type AttachMediaOptions = {
+ paths: Paths;
+ slug: string;
+ channelConfig: ChannelConfig;
+ source: string;
+ items?: AttachItem[];
+ match?: string;
+ createRecords?: boolean;
+ replace?: boolean;
+ dryRun?: boolean;
+ onLog: (line: string) => void;
+ signal?: AbortSignal;
+ drainSignal?: AbortSignal;
+ deps?: {
+ now?: () => Date;
+ probeDuration?: (file: string) => Promise<number | null>;
+ openSource?: (source: string) => Promise<ArchiveSource>;
+ };
+};
+
+export type AttachMediaResult = {
+ counts: Record<AttachRowClass, number>;
+ attached: string[];
+ created: string[];
+ failed: { id: string; error: string }[];
+ notHeld: string[];
+ lost: string[];
+ unmatched: string[];
+ ambiguous: string[];
+ alreadyAttached: string[];
+ heldWithoutMedia: string[] | null;
+ stopped?: string;
+ dryRun: boolean;
+};
+
+const LINE = (s: string) => (s.endsWith("\n") ? s : `${s}\n`);
+
+async function heldIds(dataDir: string): Promise<Set<string>> {
+ try {
+ const entries = await readdir(dataDir, { withFileTypes: true });
+ return new Set(entries.filter((d) => d.isDirectory() && !d.name.startsWith(".")).map((d) => d.name));
+ } catch (err) {
+ if ((err as NodeJS.ErrnoException).code === "ENOENT") return new Set();
+ throw err;
+ }
+}
+
+function extOf(p: string): string {
+ const dot = p.lastIndexOf(".");
+ return dot < 0 ? "" : p.slice(dot + 1).toLowerCase();
+}
+
+async function storeRootExists(root: string): Promise<boolean> {
+ const st = await stat(root).catch(() => null);
+ return !!st?.isDirectory();
+}
+
+// The record's metadata.info.json, for a video the channel does not hold.
+async function recordInfo(
+ source: ArchiveSource,
+ row: AttachRow,
+ opts: AttachMediaOptions,
+ stagedMedia: string,
+): Promise<Record<string, unknown>> {
+ const folder = row.source;
+ if (folder?.infoJson) {
+ const parsed = JSON.parse(await source.readText(folder.infoJson)) as unknown;
+ if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) {
+ throw new Error(`${folder.infoJson.path} is not a JSON object`);
+ }
+ const info = parsed as Record<string, unknown>;
+ if (info.id !== row.id) {
+ throw new Error(`${folder.infoJson.path} is for "${String(info.id)}", not ${row.id}`);
+ }
+ return info;
+ }
+ const descriptionText = folder?.descriptionTxt ? await source.readText(folder.descriptionTxt) : undefined;
+ const probe =
+ opts.deps?.probeDuration ??
+ ((f: string) => probeMediaDurationSec({ ffprobeBin: opts.paths.ffprobeBin, file: f, ...(opts.signal ? { signal: opts.signal } : {}) }));
+ const durationSec = await probe(stagedMedia);
+ return archiveFolderInfoJson({
+ id: row.id as string,
+ folderName: folder?.name ?? "",
+ ...(descriptionText !== undefined ? { descriptionText } : {}),
+ ...(opts.channelConfig.platform ? { platform: opts.channelConfig.platform } : {}),
+ durationSec,
+ });
+}
+
+function logPlan(plan: AttachPlan, onLog: (s: string) => void): void {
+ for (const r of plan.rows) {
+ const what = r.media ? ` ← ${r.media.path} (${formatBytes(r.media.size)})` : "";
+ const note =
+ r.class === "ambiguous" ? ` — ${r.candidates?.length ?? 0} video files: name one with "items"` : "";
+ onLog(LINE(`${r.class.padEnd(16)} ${r.id ?? "(no id)"}${what}${r.class === "attach" && r.replacing ? " [replacing]" : ""}${!r.media ? ` ${r.folder}` : ""}${note}`));
+ }
+ if (plan.heldWithoutMedia) {
+ onLog(LINE(`held, no media in the archive: ${plan.heldWithoutMedia.length}${plan.heldWithoutMedia.length ? ` — ${plan.heldWithoutMedia.join(", ")}` : ""}`));
+ }
+}
+
+export async function attachMedia(opts: AttachMediaOptions): Promise<AttachMediaResult> {
+ const onLog = (s: string) => opts.onLog(LINE(s));
+ const now = opts.deps?.now ?? (() => new Date());
+ let match: RegExp | undefined;
+ if (opts.match !== undefined) {
+ try {
+ match = new RegExp(opts.match, "i");
+ } catch (e) {
+ throw new Error(`"match" is not a valid regex: ${(e as Error).message}`);
+ }
+ }
+ const source = await (opts.deps?.openSource ?? ((s) => openArchiveSource(s)))(opts.source);
+ onLog(`${source.kind} ${source.archive}: ${source.files.length} files`);
+
+ const channelDir = path.join(opts.paths.channelsDir, opts.slug);
+ const dataDir = path.join(channelDir, "data");
+ const held = await heldIds(dataDir);
+ const attached = new Set<string>();
+ const pointers = new Map<string, Awaited<ReturnType<typeof loadSavedVideo>>>();
+ for (const id of held) {
+ const p = await loadSavedVideo(path.join(dataDir, id));
+ if (p) {
+ attached.add(id);
+ pointers.set(id, p);
+ }
+ }
+
+ const folders = groupArchiveFolders(source.files);
+ const plan = planAttach({
+ folders,
+ held,
+ attached,
+ ...(opts.createRecords ? { createRecords: true } : {}),
+ ...(opts.replace ? { replace: true } : {}),
+ ...(match ? { match } : {}),
+ ...(opts.items ? { items: opts.items, files: source.files } : {}),
+ });
+ logPlan(plan, onLog);
+
+ const ids = (c: AttachRowClass) => plan.rows.filter((r) => r.class === c).map((r) => r.id ?? r.folder);
+ const result: AttachMediaResult = {
+ counts: countAttachRows(plan.rows),
+ attached: [],
+ created: [],
+ failed: [],
+ notHeld: ids("not-held"),
+ lost: ids("lost"),
+ unmatched: ids("unmatched"),
+ ambiguous: ids("ambiguous"),
+ alreadyAttached: ids("already-attached"),
+ heldWithoutMedia: plan.heldWithoutMedia,
+ dryRun: !!opts.dryRun,
+ };
+ const work = plan.rows.filter((r) => r.class === "attach" || r.class === "create");
+ const total = work.reduce((s, r) => s + (r.media?.size ?? 0), 0);
+ onLog(`${work.length} to attach (${formatBytes(total)}): ${result.counts.attach} held, ${result.counts.create} new records`);
+ if (opts.dryRun || work.length === 0) {
+ onLog(`summary: ${JSON.stringify(summaryOf(result))}`);
+ return result;
+ }
+
+ const storeRoot = savedVideoRoot(opts.paths, opts.channelConfig);
+ if (!(await storeRootExists(storeRoot))) {
+ throw new Error(`the saved-video store ${storeRoot} is not there (is its drive mounted?) — nothing attached`);
+ }
+
+ let n = 0;
+ for (const row of work) {
+ n += 1;
+ if (opts.signal?.aborted) {
+ result.stopped = "cancelled";
+ break;
+ }
+ if (opts.drainSignal?.aborted) {
+ result.stopped = "drained";
+ onLog(`Drained: ${work.length - n + 1} left for the next run.`);
+ break;
+ }
+ const id = row.id as string;
+ const media = row.media as SourceFile;
+ const storeDir = savedVideoDir(opts.paths, opts.channelConfig, opts.slug, id);
+ try {
+ await assertSavedVideosStoreWritable(opts.paths, storeDir);
+ } catch (err) {
+ result.stopped = (err as Error).message;
+ onLog(`Stopped: ${result.stopped}`);
+ break;
+ }
+ const free = await getFreeBytes(storeRoot);
+ if (free < media.size + ATTACH_MEDIA_DISK_MARGIN_BYTES) {
+ result.stopped = `the store's disk has ${formatBytes(free)} free; ${id} needs ${formatBytes(media.size)} plus ${formatBytes(ATTACH_MEDIA_DISK_MARGIN_BYTES)}`;
+ onLog(`Stopped: ${result.stopped}. A re-run resumes.`);
+ break;
+ }
+ const slugDir = path.dirname(storeDir);
+ const staging = path.join(slugDir, `.${id}.attach-${process.pid}.part`);
+ const ext = extOf(media.path);
+ const videoDir = path.join(dataDir, id);
+ try {
+ onLog(`[${n}/${work.length}] ${id}: copying ${media.path} (${formatBytes(media.size)})`);
+ await mkdir(slugDir, { recursive: true });
+ await rm(staging, { force: true });
+ const copied = await source.copyOut(media, staging, opts.signal);
+ if (row.class === "create") {
+ const info = await recordInfo(source, row, opts, staging);
+ await mkdir(videoDir, { recursive: true });
+ await withMetadataHistory(videoDir, { by: "local-archive", requestedBy: ATTACH_MEDIA_REQUESTER, onLog }, () =>
+ writeJsonAtomic(path.join(videoDir, "metadata.info.json"), info, { indent: 0, newline: false }),
+ );
+ const extractor = typeof info.extractor_key === "string" ? info.extractor_key.toLowerCase() : null;
+ if (extractor) {
+ await appendFile(path.join(channelDir, "archive"), `${extractor} ${id}\n`);
+ }
+ result.created.push(id);
+ onLog(`${id}: record written (${typeof info.title === "string" ? info.title : id})`);
+ }
+ const previous = pointers.get(id) ?? null;
+ const origin: SavedVideoOrigin = {
+ requestedBy: ATTACH_MEDIA_REQUESTER,
+ kind: "local-archive",
+ archive: source.archive,
+ entry: media.path,
+ sha256: copied.sha256,
+ attachedAt: now().toISOString(),
+ };
+ const pointer = await persistSourceVideo({
+ videoDir,
+ sourceFilename: `source-media.${ext}`,
+ sourcePath: staging,
+ storeDir,
+ keepReason: "pin",
+ origin,
+ sha256: copied.sha256,
+ paths: opts.paths,
+ });
+ if (previous && savedVideoPath(previous) !== savedVideoPath(pointer)) {
+ await rm(savedVideoPath(previous), { force: true });
+ onLog(`${id}: the replaced container ${previous.file} removed`);
+ }
+ result.attached.push(id);
+ onLog(`${id}: attached ${pointer.file} (${formatBytes(pointer.bytes)}, sha256 ${copied.sha256.slice(0, 12)}…)`);
+ } catch (err) {
+ await rm(staging, { force: true }).catch(() => {});
+ if (opts.signal?.aborted) {
+ result.stopped = "cancelled";
+ break;
+ }
+ result.failed.push({ id, error: (err as Error).message });
+ onLog(`${id}: FAILED — ${(err as Error).message}`);
+ }
+ }
+ onLog(`summary: ${JSON.stringify(summaryOf(result))}`);
+ return result;
+}
+
+export function summaryOf(r: AttachMediaResult): Record<string, unknown> {
+ return {
+ dryRun: r.dryRun,
+ counts: r.counts,
+ attached: r.attached.length,
+ created: r.created,
+ failed: r.failed,
+ notHeld: r.notHeld,
+ lost: r.lost.length,
+ unmatched: r.unmatched,
+ ambiguous: r.ambiguous,
+ alreadyAttached: r.alreadyAttached.length,
+ heldWithoutMedia: r.heldWithoutMedia,
+ ...(r.stopped ? { stopped: r.stopped } : {}),
+ };
+}
diff --git a/common/controller/preparePlayable.test.ts b/common/controller/preparePlayable.test.ts
@@ -0,0 +1,275 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import { createHash } from "node:crypto";
+import { copyFile, mkdir, mkdtemp, readFile, rm, stat, writeFile } from "node:fs/promises";
+import { tmpdir } from "node:os";
+import path from "node:path";
+import { execa } from "execa";
+import parseTorrent from "parse-torrent";
+import type { Paths } from "../lib/paths";
+import type { ChannelConfig } from "../lib/channelConfig";
+import { loadSavedVideo } from "../lib/savedVideo-server";
+import {
+ loadPlayableManifest,
+ makePlayableTorrent,
+ preparePlayable,
+ type PreparePlayableOptions,
+} from "./preparePlayable";
+
+// Run with: pnpm --filter yt-dlp-transcript-common exec tsx --test controller/preparePlayable.test.ts
+//
+// Fixture media made here by ffmpeg (a few seconds of test pattern and tone),
+// in a temp dir. Skipped where ffmpeg/ffprobe are not installed.
+
+const SLUG = "demo";
+const have = async (bin: string) => (await execa(bin, ["-version"], { reject: false }).catch(() => null))?.exitCode === 0;
+const tools = (await have("ffmpeg")) && (await have("ffprobe"));
+
+async function make(file: string, vArgs: string[], aArgs: string[]): Promise<void> {
+ await execa("ffmpeg", [
+ "-nostdin", "-hide_banner", "-loglevel", "error", "-y",
+ "-f", "lavfi", "-i", "testsrc=d=2:s=96x64:r=10",
+ "-f", "lavfi", "-i", "sine=d=2:f=440",
+ ...vArgs, ...aArgs, "-shortest", file,
+ ]);
+}
+
+const sha = async (f: string) => createHash("sha256").update(await readFile(f)).digest("hex");
+
+// Top-level MP4 atoms in order — "moov" before "mdat" is a faststart file.
+async function topAtoms(file: string): Promise<string[]> {
+ const b = await readFile(file);
+ const out: string[] = [];
+ let at = 0;
+ while (at + 8 <= b.length) {
+ let size = b.readUInt32BE(at);
+ const type = b.toString("latin1", at + 4, at + 8);
+ if (size === 1) size = Number(b.readBigUInt64BE(at + 8));
+ if (size < 8) break;
+ out.push(type);
+ at += size;
+ }
+ return out;
+}
+
+async function videoFrameMd5(file: string): Promise<string> {
+ const r = await execa("ffmpeg", ["-nostdin", "-loglevel", "error", "-i", file, "-map", "0:V:0", "-c", "copy", "-f", "framemd5", "-"]);
+ // Drop the header comments (they name the container's timebase).
+ return String(r.stdout).split("\n").filter((l) => l && !l.startsWith("#")).map((l) => l.split(",").pop()).join("\n");
+}
+
+type Fixture = { dir: string; paths: Paths; root: string; opts: (o?: Partial<PreparePlayableOptions>) => PreparePlayableOptions; lines: string[] };
+
+async function withFixture(fn: (f: Fixture) => Promise<void>): Promise<void> {
+ const dir = await mkdtemp(path.join(tmpdir(), "ttb-playable-"));
+ const paths = {
+ transcriptsDir: path.join(dir, "corpus"),
+ channelsDir: path.join(dir, "corpus", "channels"),
+ savedVideosDir: path.join(dir, "media", "saved-videos"),
+ ffmpegBin: "ffmpeg",
+ ffprobeBin: "ffprobe",
+ } as Paths;
+ const root = path.join(dir, "media", "playable");
+ const lines: string[] = [];
+ const opts = (o: Partial<PreparePlayableOptions> = {}): PreparePlayableOptions => ({
+ paths,
+ slug: SLUG,
+ channelConfig: { name: "Demo", handling: "youtube", platform: "youtube" } as unknown as ChannelConfig,
+ onLog: (l) => lines.push(l),
+ deps: { now: () => new Date("2026-10-09T12:00:00Z") },
+ ...o,
+ });
+ try {
+ await fn({ dir, paths, root, opts, lines });
+ } finally {
+ await rm(dir, { recursive: true, force: true });
+ }
+}
+
+// One saved container per id, with a pointer (sha256 recorded or not).
+async function saved(paths: Paths, id: string, src: string, ext: string, withSha = true): Promise<string> {
+ const storeDir = path.join(paths.savedVideosDir, SLUG, id);
+ await mkdir(storeDir, { recursive: true });
+ const file = path.join(storeDir, `source-media.${ext}`);
+ await copyFile(src, file);
+ const vd = path.join(paths.channelsDir, SLUG, "data", id);
+ await mkdir(vd, { recursive: true });
+ await writeFile(
+ path.join(vd, "saved-video.json"),
+ JSON.stringify({
+ storedAt: "x",
+ dir: storeDir,
+ file: `source-media.${ext}`,
+ bytes: (await stat(file)).size,
+ keepReason: "pin",
+ ...(withSha ? { sha256: await sha(file) } : {}),
+ }),
+ );
+ return file;
+}
+
+test("remuxes each container losslessly, makes its torrent, and a re-run is a no-op", { skip: !tools && "ffmpeg/ffprobe not installed" }, async () => {
+ await withFixture(async ({ dir, paths, root, opts }) => {
+ const src = path.join(dir, "src");
+ await mkdir(src);
+ await make(path.join(src, "a.mp4"), ["-c:v", "libx264", "-preset", "ultrafast"], ["-c:a", "aac"]);
+ await make(path.join(src, "b.mkv"), ["-c:v", "libvpx-vp9", "-deadline", "realtime", "-cpu-used", "8"], ["-c:a", "libopus"]);
+ await make(path.join(src, "c.webm"), ["-c:v", "libvpx-vp9", "-deadline", "realtime", "-cpu-used", "8"], ["-c:a", "libopus"]);
+ await make(path.join(src, "d.mkv"), ["-c:v", "mpeg4"], ["-c:a", "aac"]);
+ // The source mp4 has its moov at the back (ffmpeg's default).
+ assert.ok((await topAtoms(path.join(src, "a.mp4"))).indexOf("moov") > (await topAtoms(path.join(src, "a.mp4"))).indexOf("mdat"));
+
+ const aSrc = await saved(paths, "aaaaaaaaaaa", path.join(src, "a.mp4"), "mp4");
+ await saved(paths, "bbbbbbbbbbb", path.join(src, "b.mkv"), "mkv", false);
+ await saved(paths, "ccccccccccc", path.join(src, "c.webm"), "webm");
+ await saved(paths, "ddddddddddd", path.join(src, "d.mkv"), "mkv");
+ await mkdir(path.join(paths.channelsDir, SLUG, "data", "eeeeeeeeeee"), { recursive: true }); // held, no container
+
+ let remuxes = 0;
+ const counting = (o: Partial<PreparePlayableOptions> = {}) =>
+ opts({
+ root,
+ ...o,
+ deps: {
+ now: () => new Date("2026-10-09T12:00:00Z"),
+ remux: async (s, d, args) => {
+ remuxes += 1;
+ await execa("ffmpeg", ["-nostdin", "-hide_banner", "-loglevel", "error", "-y", "-i", s, ...args, d]);
+ },
+ },
+ });
+
+ const r = await preparePlayable(counting());
+ assert.deepEqual(r.prepared, ["aaaaaaaaaaa", "bbbbbbbbbbb", "ccccccccccc"]);
+ assert.deepEqual(r.unplayable.map((u) => u.id), ["ddddddddddd"]);
+ assert.match(r.unplayable[0].reason, /mpeg4/);
+ assert.deepEqual(r.failed, []);
+ assert.equal(remuxes, 3);
+
+ const m = await loadPlayableManifest(root, SLUG);
+ assert.deepEqual(Object.keys(m.entries).sort(), ["aaaaaaaaaaa", "bbbbbbbbbbb", "ccccccccccc"]);
+ const a = m.entries.aaaaaaaaaaa;
+ assert.equal(a.file, "aaaaaaaaaaa/aaaaaaaaaaa.mp4");
+ assert.equal(a.mime, "video/mp4");
+ assert.equal(a.sourceSha256, await sha(aSrc));
+ assert.equal(m.entries.bbbbbbbbbbb.ext, "mp4", "vp9/opus in mkv → mp4");
+ assert.equal(m.entries.ccccccccccc.ext, "webm", "vp9/opus in webm stays webm");
+ assert.equal(m.entries.bbbbbbbbbbb.acodec, "opus");
+
+ // Faststart: the index is in front now.
+ const atoms = await topAtoms(path.join(root, SLUG, a.file));
+ assert.ok(atoms.indexOf("moov") < atoms.indexOf("mdat"), atoms.join(","));
+ // Lossless: every video packet is bit-identical.
+ assert.equal(await videoFrameMd5(path.join(root, SLUG, a.file)), await videoFrameMd5(aSrc));
+ // Bitexact: the same source remuxes to the same bytes (so to the same infohash).
+ const again1 = path.join(dir, "again.mp4");
+ const decision = (await import("../lib/playableMedia")).decidePlayable({ formatName: "", ext: "mp4", vcodec: "h264", acodec: "aac" });
+ assert.ok(decision.playable);
+ if (decision.playable) {
+ await execa("ffmpeg", ["-nostdin", "-loglevel", "error", "-y", "-i", aSrc, ...decision.args, again1]);
+ assert.equal(await sha(again1), await sha(path.join(root, SLUG, a.file)));
+ }
+
+ // The torrent: one file, named <id>.<ext>, the manifest's infohash, and
+ // nothing that names a person, a tool or a URL.
+ const tBuf = await readFile(path.join(root, SLUG, a.torrent));
+ const t = await parseTorrent(tBuf);
+ assert.equal(t.infoHash, a.infoHash);
+ assert.equal(t.name, "aaaaaaaaaaa.mp4");
+ assert.equal(t.length, a.bytes);
+ assert.deepEqual(t.urlList, []);
+ assert.deepEqual(t.announce, []);
+ assert.equal(t.createdBy, undefined);
+ assert.equal(t.comment, undefined);
+ assert.notEqual(t.private, true);
+ for (const k of ["created by", "comment", "url-list", "announce", "private"]) {
+ assert.equal(tBuf.includes(Buffer.from(`${k.length}:${k}`)), false, `no ${k}`);
+ }
+
+ // A pointer that had no hash got one.
+ assert.match((await loadSavedVideo(path.join(paths.channelsDir, SLUG, "data", "bbbbbbbbbbb")))?.sha256 ?? "", /^[0-9a-f]{64}$/);
+
+ // Again: nothing to do, nothing remuxed, the manifest unchanged.
+ const before = await readFile(path.join(root, SLUG, "playable.json"), "utf8");
+ const again = await preparePlayable(counting());
+ assert.deepEqual(again.unchanged.sort(), ["aaaaaaaaaaa", "bbbbbbbbbbb", "ccccccccccc"]);
+ assert.deepEqual(again.prepared, []);
+ assert.equal(remuxes, 3);
+ assert.equal(await readFile(path.join(root, SLUG, "playable.json"), "utf8"), before);
+ assert.deepEqual(again.noContainer, []);
+ });
+});
+
+test("each remux goes through the heavy slot's gate when the checkout has one", { skip: !tools && "ffmpeg/ffprobe not installed" }, async () => {
+ await withFixture(async ({ dir, paths, root, opts }) => {
+ await make(path.join(dir, "a.mp4"), ["-c:v", "libx264", "-preset", "ultrafast"], ["-c:a", "aac"]);
+ await saved(paths, "aaaaaaaaaaa", path.join(dir, "a.mp4"), "mp4");
+ // A stand-in for scripts/queue-lock.mjs: records its argv, runs the rest.
+ const repo = path.join(dir, "repo");
+ await mkdir(path.join(repo, "scripts"), { recursive: true });
+ const marker = path.join(dir, "gate.json");
+ await writeFile(
+ path.join(repo, "scripts", "queue-lock.mjs"),
+ `import { writeFileSync } from "node:fs";\nimport { spawnSync } from "node:child_process";\n` +
+ `const argv = process.argv.slice(2);\nwriteFileSync(${JSON.stringify(marker)}, JSON.stringify(argv));\n` +
+ `const at = argv.indexOf("--");\nconst r = spawnSync(argv[at + 1], argv.slice(at + 2), { stdio: "inherit" });\nprocess.exit(r.status ?? 1);\n`,
+ );
+ const r = await preparePlayable(opts({ root, paths: { ...paths, monorepoRoot: repo } as Paths }));
+ assert.deepEqual(r.prepared, ["aaaaaaaaaaa"]);
+ const argv = JSON.parse(await readFile(marker, "utf8")) as string[];
+ assert.deepEqual(argv.slice(0, 3), ["--heavy", "--", "ffmpeg"]);
+ });
+});
+
+test("dryRun decides and writes nothing", { skip: !tools && "ffmpeg/ffprobe not installed" }, async () => {
+ await withFixture(async ({ dir, paths, root, opts }) => {
+ await make(path.join(dir, "a.mp4"), ["-c:v", "libx264", "-preset", "ultrafast"], ["-c:a", "aac"]);
+ await saved(paths, "aaaaaaaaaaa", path.join(dir, "a.mp4"), "mp4", false);
+ const r = await preparePlayable(opts({ root, dryRun: true }));
+ assert.deepEqual(r.prepared, ["aaaaaaaaaaa"]);
+ await assert.rejects(stat(root));
+ assert.equal((await loadSavedVideo(path.join(paths.channelsDir, SLUG, "data", "aaaaaaaaaaa")))?.sha256, undefined);
+ });
+});
+
+test("the default root is playable/ beside the store's real directory; an absent drive refuses", async () => {
+ await withFixture(async ({ dir, paths, opts }) => {
+ await mkdir(paths.savedVideosDir, { recursive: true });
+ const r = await preparePlayable(opts({ ids: [] }));
+ assert.equal(r.root, path.join(dir, "media", "playable"));
+ await assert.rejects(
+ preparePlayable(opts({ root: path.join(dir, "not-mounted", "playable"), ids: [] })),
+ /is not there/,
+ );
+ });
+});
+
+test("torrent determinism: same bytes + piece length → same infohash, whatever the trackers or the path", async () => {
+ const dir = await mkdtemp(path.join(tmpdir(), "ttb-torrent-"));
+ try {
+ const bytes = Buffer.from(Array.from({ length: 700_000 }, (_, i) => (i * 31) & 0xff));
+ const f1 = path.join(dir, "one.bin");
+ const f2 = path.join(dir, "elsewhere.bin");
+ await writeFile(f1, bytes);
+ await writeFile(f2, bytes);
+ const base = { name: "x.mp4", bytes: bytes.length };
+ const a = await makePlayableTorrent({ ...base, file: f1 });
+ const b = await makePlayableTorrent({ ...base, file: f2 });
+ const c = await makePlayableTorrent({ ...base, file: f1, trackers: ["wss://tracker.example/announce", "udp://t.example:6969"] });
+ const d = await makePlayableTorrent({ ...base, file: f1, pieceLength: 512 * 1024 });
+ assert.match(a.infoHash, /^[0-9a-f]{40}$/);
+ assert.equal(a.infoHash, b.infoHash, "the path does not matter");
+ assert.equal(a.infoHash, c.infoHash, "the trackers are outside the info dict");
+ assert.notEqual(a.infoHash, d.infoHash, "the piece length is inside it");
+ assert.equal(a.pieceLength, 256 * 1024);
+ assert.deepEqual(a.torrent, b.torrent, "fixed creation date: the same .torrent byte for byte");
+ const parsed = await parseTorrent(c.torrent);
+ assert.deepEqual(parsed.announce, ["wss://tracker.example/announce", "udp://t.example:6969"]);
+ // Different bytes, different hash.
+ bytes[0] ^= 1;
+ await writeFile(f2, bytes);
+ assert.notEqual((await makePlayableTorrent({ ...base, file: f2 })).infoHash, a.infoHash);
+ } finally {
+ await rm(dir, { recursive: true, force: true });
+ }
+});
diff --git a/common/controller/preparePlayable.ts b/common/controller/preparePlayable.ts
@@ -0,0 +1,361 @@
+// PLAYABLE COPIES AND THEIR TORRENTS (release 21 D3, the `prepare-playable`
+// job).
+//
+// For each of a channel's videos with a saved source container: a browser-
+// playable copy made by a LOSSLESS remux (lib/playableMedia.ts decides the
+// container and the ffmpeg arguments), and one single-file torrent of that
+// copy. Written under the playable root —
+//
+// <root>/<slug>/<id>/<id>.<ext> the copy
+// <root>/<slug>/<id>/<id>.torrent its torrent
+// <root>/<slug>/playable.json the manifest (lib/playableMedia.ts)
+//
+// — where <root> is, unless named, `playable/` BESIDE the saved-video store's
+// real directory (the store is on the archive drive; so are the copies).
+//
+// KEYED BY THE SOURCE'S SHA-256, so a re-run is a no-op: an entry whose
+// `sourceSha256` matches the pointer's and whose files are on disk at their
+// recorded size is left alone. A pointer with no hash (a download's, before
+// the backup step reached it) is hashed once and the hash recorded on the
+// pointer, as the backup step would.
+//
+// THE TORRENT. `create-torrent`, from the WebTorrent ecosystem: one file,
+// named `<id>.<ext>`; piece length scaled to the size (256 KiB–1 MiB); no web
+// seed (`urlList`), no comment, no `created by`, a fixed creation date — the
+// .torrent names nobody and no private URL. The announce list is the
+// `trackers` parameter (none by default): it is outside the info dict, so the
+// infohash depends only on the bytes and the piece length, and a site can
+// announce the same infohash to other trackers later without a re-hash.
+//
+// Each copy is written to a `.part` and renamed; the manifest is rewritten
+// after each video, so a cancelled or drained run resumes where it stopped.
+
+/// <reference path="../types/webtorrent-modules.d.ts" />
+import path from "node:path";
+import { createReadStream } from "node:fs";
+import { mkdir, readdir, readFile, realpath, rename, rm, stat, writeFile } from "node:fs/promises";
+import { createHash } from "node:crypto";
+import { execa } from "execa";
+import createTorrent from "create-torrent";
+import parseTorrent from "parse-torrent";
+import type { Paths } from "../lib/paths";
+import type { ChannelConfig } from "../lib/channelConfig";
+import { savedVideoPath, savedVideoRoot } from "../lib/savedVideo";
+import { loadSavedVideo, updateSavedVideoChecksum } from "../lib/savedVideo-server";
+import { writeJsonAtomic } from "../lib/jsonFile-server";
+import { getFreeBytes } from "../lib/diskSpace";
+import { formatBytes } from "../lib/format";
+import {
+ PLAYABLE_MANIFEST_FILENAME,
+ decidePlayable,
+ parsePlayableManifest,
+ parseProbe,
+ pieceLengthFor,
+ type MediaProbe,
+ type PlayableEntry,
+ type PlayableManifest,
+} from "../lib/playableMedia";
+
+// The creation date every torrent carries: fixed, so the same copy makes the
+// same .torrent file byte for byte (the infohash never depended on it).
+export const PLAYABLE_TORRENT_CREATION_DATE = new Date("2026-01-01T00:00:00Z");
+
+export const PLAYABLE_DISK_MARGIN_BYTES = 2 * 1024 ** 3;
+
+export async function defaultPlayableRoot(
+ paths: Paths,
+ config: Pick<ChannelConfig, "savedVideosDir"> | null,
+): Promise<string> {
+ const store = savedVideoRoot(paths, config);
+ const real = await realpath(store).catch(() => store);
+ return path.join(path.dirname(real), "playable");
+}
+
+export function playableManifestPath(root: string, slug: string): string {
+ return path.join(root, slug, PLAYABLE_MANIFEST_FILENAME);
+}
+
+export async function loadPlayableManifest(root: string, slug: string): Promise<PlayableManifest> {
+ try {
+ return parsePlayableManifest(JSON.parse(await readFile(playableManifestPath(root, slug), "utf8")), slug);
+ } catch {
+ return parsePlayableManifest(null, slug);
+ }
+}
+
+// --- the torrent ------------------------------------------------------------
+
+export type TorrentMade = { torrent: Buffer; infoHash: string; pieceLength: number };
+
+// One single-file torrent of `file`, named `name`. Deterministic: the same
+// bytes and piece length give the same infohash, whatever `trackers` is.
+export async function makePlayableTorrent(opts: {
+ file: string;
+ name: string;
+ bytes: number;
+ trackers?: readonly string[];
+ pieceLength?: number;
+}): Promise<TorrentMade> {
+ const pieceLength = opts.pieceLength ?? pieceLengthFor(opts.bytes);
+ // What is NOT passed matters as much as what is. `createdBy`, `comment`
+ // and `urlList` absent: create-torrent then writes no such key. `private`
+ // absent: even `false` would put `private: 0` INSIDE the info dict and so
+ // change the infohash — the one thing that must not move when the torrent
+ // goes public later.
+ const torrent = await new Promise<Buffer>((resolve, reject) => {
+ createTorrent(
+ opts.file,
+ {
+ name: opts.name,
+ pieceLength,
+ creationDate: PLAYABLE_TORRENT_CREATION_DATE,
+ // A list, even an empty one, not create-torrent's default public
+ // trackers: the pilot announces only where it is told to. A fresh
+ // array each call (create-torrent appends to the one it is given).
+ announceList: opts.trackers && opts.trackers.length ? [[...opts.trackers]] : [],
+ },
+ (err, buf) => (err ? reject(err) : resolve(Buffer.from(buf))),
+ );
+ });
+ const parsed = await parseTorrent(torrent);
+ if (!parsed.infoHash) throw new Error(`no infohash in the torrent made for ${opts.name}`);
+ return { torrent, infoHash: parsed.infoHash, pieceLength };
+}
+
+// --- the run ----------------------------------------------------------------
+
+export type PreparePlayableOptions = {
+ paths: Paths;
+ slug: string;
+ channelConfig: ChannelConfig;
+ root?: string;
+ ids?: string[];
+ trackers?: string[];
+ dryRun?: boolean;
+ onLog: (line: string) => void;
+ signal?: AbortSignal;
+ drainSignal?: AbortSignal;
+ deps?: {
+ now?: () => Date;
+ probe?: (file: string) => Promise<MediaProbe>;
+ remux?: (src: string, dest: string, args: string[], signal?: AbortSignal) => Promise<void>;
+ };
+};
+
+export type PreparePlayableResult = {
+ root: string;
+ prepared: string[];
+ unchanged: string[];
+ unplayable: { id: string; reason: string }[];
+ noContainer: string[];
+ failed: { id: string; error: string }[];
+ stopped?: string;
+ dryRun: boolean;
+};
+
+async function sha256File(file: string, signal?: AbortSignal): Promise<string> {
+ const h = createHash("sha256");
+ for await (const chunk of createReadStream(file, { highWaterMark: 1 << 20, ...(signal ? { signal } : {}) })) {
+ h.update(chunk as Buffer);
+ }
+ return h.digest("hex");
+}
+
+async function ffprobe(paths: Paths, file: string, signal?: AbortSignal): Promise<MediaProbe> {
+ const r = await execa(
+ paths.ffprobeBin,
+ ["-v", "error", "-show_streams", "-show_format", "-of", "json", file],
+ { ...(signal ? { cancelSignal: signal } : {}) },
+ );
+ return parseProbe(JSON.parse(String(r.stdout)), path.extname(file));
+}
+
+// EACH REMUX TAKES THE HEAVY SLOT (release 19 B1, scripts/queue-lock.mjs
+// --heavy — the gate `pnpm heavy` and a publish stage's `next build` use):
+// one heavy job machine-wide, above the memory floor. Per video, not per run,
+// so a 36 GB channel interleaves with builds and e2e instead of holding the
+// slot for hours. The gate forwards a cancel to ffmpeg. A checkout without
+// the gate (a test's temp root) runs ffmpeg directly; HEAVY=0 bypasses it.
+async function ffmpegRemux(paths: Paths, src: string, dest: string, args: string[], signal?: AbortSignal): Promise<void> {
+ const ffmpegArgs = ["-nostdin", "-hide_banner", "-loglevel", "error", "-y", "-i", src, ...args, dest];
+ const gate = paths.monorepoRoot ? path.join(paths.monorepoRoot, "scripts", "queue-lock.mjs") : null;
+ const gated = gate !== null && (await stat(gate).then((s) => s.isFile()).catch(() => false));
+ await execa(
+ gated ? process.execPath : paths.ffmpegBin,
+ gated ? [gate, "--heavy", "--", paths.ffmpegBin, ...ffmpegArgs] : ffmpegArgs,
+ { ...(signal ? { cancelSignal: signal } : {}) },
+ );
+}
+
+async function sizeOf(file: string): Promise<number | null> {
+ const st = await stat(file).catch(() => null);
+ return st?.isFile() ? st.size : null;
+}
+
+async function heldWithPointers(dataDir: string): Promise<string[]> {
+ const entries = await readdir(dataDir, { withFileTypes: true }).catch((err: NodeJS.ErrnoException) => {
+ if (err.code === "ENOENT") return [];
+ throw err;
+ });
+ const out: string[] = [];
+ for (const d of entries) {
+ if (!d.isDirectory() || d.name.startsWith(".")) continue;
+ if (await loadSavedVideo(path.join(dataDir, d.name))) out.push(d.name);
+ }
+ return out.sort();
+}
+
+export async function preparePlayable(opts: PreparePlayableOptions): Promise<PreparePlayableResult> {
+ const onLog = (s: string) => opts.onLog(s.endsWith("\n") ? s : `${s}\n`);
+ const now = opts.deps?.now ?? (() => new Date());
+ const probe = opts.deps?.probe ?? ((f: string) => ffprobe(opts.paths, f, opts.signal));
+ const remux =
+ opts.deps?.remux ??
+ ((src: string, dest: string, args: string[], signal?: AbortSignal) => ffmpegRemux(opts.paths, src, dest, args, signal));
+ const root = opts.root ?? (await defaultPlayableRoot(opts.paths, opts.channelConfig));
+ const result: PreparePlayableResult = {
+ root,
+ prepared: [],
+ unchanged: [],
+ unplayable: [],
+ noContainer: [],
+ failed: [],
+ dryRun: !!opts.dryRun,
+ };
+ // The root's parent must exist: a copy aimed at an unmounted drive is never
+ // materialised on the root filesystem by a mkdir.
+ const parent = await stat(path.dirname(root)).catch(() => null);
+ if (!parent?.isDirectory()) {
+ throw new Error(`${path.dirname(root)} is not there (is its drive mounted?) — the playable root ${root} cannot be made`);
+ }
+ const dataDir = path.join(opts.paths.channelsDir, opts.slug, "data");
+ const ids = opts.ids ?? (await heldWithPointers(dataDir));
+ const slugDir = path.join(root, opts.slug);
+ const manifest = await loadPlayableManifest(root, opts.slug);
+ onLog(`${ids.length} video(s) with a saved container; playable root ${root}`);
+
+ let n = 0;
+ for (const id of ids) {
+ n += 1;
+ if (opts.signal?.aborted) {
+ result.stopped = "cancelled";
+ break;
+ }
+ if (opts.drainSignal?.aborted) {
+ result.stopped = "drained";
+ onLog(`Drained: ${ids.length - n + 1} left for the next run.`);
+ break;
+ }
+ const videoDir = path.join(dataDir, id);
+ const pointer = await loadSavedVideo(videoDir);
+ const src = pointer ? savedVideoPath(pointer) : null;
+ const srcBytes = src ? await sizeOf(src) : null;
+ if (!pointer || !src || srcBytes === null) {
+ result.noContainer.push(id);
+ onLog(`${id}: no saved container${pointer ? ` (${src} is missing)` : ""}`);
+ continue;
+ }
+ try {
+ let sourceSha256 = pointer.sha256;
+ if (!sourceSha256) {
+ onLog(`${id}: hashing ${pointer.file} (${formatBytes(srcBytes)})`);
+ sourceSha256 = await sha256File(src, opts.signal);
+ if (!opts.dryRun) await updateSavedVideoChecksum(videoDir, sourceSha256);
+ }
+ const prior = manifest.entries[id];
+ if (prior && prior.sourceSha256 === sourceSha256) {
+ const [fb, tb] = await Promise.all([
+ sizeOf(path.join(slugDir, prior.file)),
+ sizeOf(path.join(slugDir, prior.torrent)),
+ ]);
+ if (fb === prior.bytes && tb !== null) {
+ result.unchanged.push(id);
+ continue;
+ }
+ }
+ const decision = decidePlayable(await probe(src));
+ if (!decision.playable) {
+ result.unplayable.push({ id, reason: decision.reason });
+ onLog(`${id}: not playable — ${decision.reason}`);
+ continue;
+ }
+ if (opts.dryRun) {
+ onLog(`[${n}/${ids.length}] ${id}: would remux — ${decision.reason}`);
+ result.prepared.push(id);
+ continue;
+ }
+ const free = await getFreeBytes(path.dirname(root));
+ if (free < srcBytes + PLAYABLE_DISK_MARGIN_BYTES) {
+ result.stopped = `the playable root's disk has ${formatBytes(free)} free; ${id} needs about ${formatBytes(srcBytes)} plus ${formatBytes(PLAYABLE_DISK_MARGIN_BYTES)}`;
+ onLog(`Stopped: ${result.stopped}. A re-run resumes.`);
+ break;
+ }
+ const outDir = path.join(slugDir, id);
+ await mkdir(outDir, { recursive: true });
+ const name = `${id}.${decision.ext}`;
+ const out = path.join(outDir, name);
+ const part = path.join(outDir, `.${name}.part`);
+ onLog(`[${n}/${ids.length}] ${id}: ${decision.reason}`);
+ await rm(part, { force: true });
+ try {
+ await remux(src, part, decision.args, opts.signal);
+ await rename(part, out);
+ } catch (err) {
+ await rm(part, { force: true }).catch(() => {});
+ throw err;
+ }
+ // A copy of another extension from an earlier run is not this video's any more.
+ for (const f of await readdir(outDir)) {
+ if (f !== name && f !== `${id}.torrent` && f.startsWith(`${id}.`)) await rm(path.join(outDir, f), { force: true });
+ }
+ const bytes = (await sizeOf(out)) ?? 0;
+ const made = await makePlayableTorrent({
+ file: out,
+ name,
+ bytes,
+ ...(opts.trackers ? { trackers: opts.trackers } : {}),
+ });
+ const torrentPath = path.join(outDir, `${id}.torrent`);
+ await writeFile(`${torrentPath}.part`, made.torrent);
+ await rename(`${torrentPath}.part`, torrentPath);
+ const probed = await probe(out).catch(() => null);
+ const entry: PlayableEntry = {
+ sourceSha256,
+ file: `${id}/${name}`,
+ torrent: `${id}/${id}.torrent`,
+ ext: decision.ext,
+ mime: decision.mime,
+ bytes,
+ infoHash: made.infoHash,
+ pieceLength: made.pieceLength,
+ vcodec: probed?.vcodec ?? "",
+ acodec: probed?.acodec ?? null,
+ preparedAt: now().toISOString(),
+ };
+ manifest.entries[id] = entry;
+ await writeJsonAtomic(playableManifestPath(root, opts.slug), manifest);
+ result.prepared.push(id);
+ onLog(`${id}: ${name} ${formatBytes(bytes)}, infohash ${made.infoHash}, ${made.pieceLength / 1024} KiB pieces`);
+ } catch (err) {
+ if (opts.signal?.aborted) {
+ result.stopped = "cancelled";
+ break;
+ }
+ result.failed.push({ id, error: (err as Error).message });
+ onLog(`${id}: FAILED — ${(err as Error).message}`);
+ }
+ }
+ onLog(
+ `summary: ${JSON.stringify({
+ dryRun: result.dryRun,
+ root,
+ prepared: result.prepared.length,
+ unchanged: result.unchanged.length,
+ unplayable: result.unplayable,
+ noContainer: result.noContainer,
+ failed: result.failed,
+ ...(result.stopped ? { stopped: result.stopped } : {}),
+ })}`,
+ );
+ return result;
+}
diff --git a/common/controller/seeder.test.ts b/common/controller/seeder.test.ts
@@ -0,0 +1,192 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
+import { tmpdir } from "node:os";
+import path from "node:path";
+import WebTorrent from "webtorrent";
+import type { Paths } from "../lib/paths";
+import { DEFAULT_SEEDER_SETTINGS, type SeederSettings } from "../lib/seederSettings";
+import { makePlayableTorrent } from "./preparePlayable";
+import { interfacePresent, mergeScrapes, runSeeder, startTracker, type SeedTarget } from "./seeder";
+
+// Run with: pnpm --filter yt-dlp-transcript-common exec tsx --test controller/seeder.test.ts
+//
+// SHORT LOCALHOST INTEGRATION: a tracker on 127.0.0.1, the seeder, and a
+// leecher in this process; a few hundred KB of synthetic bytes. Nothing
+// listens on anything but loopback, no STUN server is asked for an address,
+// and everything is torn down at the end.
+
+const ID = "aaaaaaaaaaa";
+
+async function until(pred: () => boolean, ms: number, what: string): Promise<void> {
+ const t0 = Date.now();
+ while (!pred()) {
+ if (Date.now() - t0 > ms) throw new Error(`timed out waiting for ${what}`);
+ await new Promise((r) => setTimeout(r, 100));
+ }
+}
+
+const newClient = () =>
+ new WebTorrent({
+ dht: false,
+ lsd: false,
+ natUpnp: false,
+ natPmp: false,
+ utp: false,
+ webSeeds: false,
+ tracker: { rtcConfig: { iceServers: [] } },
+ });
+
+const destroy = (c: WebTorrent) =>
+ c.destroyed ? Promise.resolve() : new Promise<void>((resolve) => c.destroy(() => resolve()));
+
+async function fixture(trackerUrl: (port: number) => string) {
+ const dir = await mkdtemp(path.join(tmpdir(), "ttb-seeder-"));
+ const seedDir = path.join(dir, "playable", "demo", ID);
+ await mkdir(seedDir, { recursive: true });
+ const bytes = Buffer.from(Array.from({ length: 600_000 }, (_, i) => (i * 13 + 7) & 0xff));
+ const file = path.join(seedDir, `${ID}.mp4`);
+ await writeFile(file, bytes);
+ let allowed = new Set<string>();
+ const tracker = await startTracker({ host: "127.0.0.1", port: 0, allowed: () => allowed });
+ const made = await makePlayableTorrent({ file, name: `${ID}.mp4`, bytes: bytes.length, trackers: [trackerUrl(tracker.port)] });
+ allowed = new Set([made.infoHash]);
+ const torrentPath = path.join(seedDir, `${ID}.torrent`);
+ await writeFile(torrentPath, made.torrent);
+ const target: SeedTarget = { slug: "demo", id: ID, infoHash: made.infoHash, torrentPath, file, dir: seedDir, bytes: bytes.length };
+ return { dir, bytes, tracker, made, target };
+}
+
+function settings(): SeederSettings {
+ return { ...DEFAULT_SEEDER_SETTINGS, sites: ["demo-site"], maxConnections: 10 };
+}
+
+function leechTo(peer: WebTorrent, torrent: Buffer, out: string, ms: number, log: string[]) {
+ const t = peer.add(torrent, { path: out });
+ return {
+ t,
+ done: new Promise<void>((resolve, reject) => {
+ t.once("done", () => resolve());
+ t.once("error", reject);
+ setTimeout(() => reject(new Error(`the leecher did not finish; seeder log: ${log.join(" | ")}`)), ms).unref();
+ }),
+ };
+}
+
+test("unit: scrape answers merge by max, and an absent interface is absent", () => {
+ const a = new Map([["h", { complete: 2, incomplete: 0 }]]);
+ const b = new Map([["h", { complete: 1, incomplete: 3 }]]);
+ assert.deepEqual(mergeScrapes([a, null, b], "h"), { complete: 2, incomplete: 3 });
+ assert.equal(mergeScrapes([null], "h"), null);
+ assert.equal(interfacePresent("wg0", { lo: [{ address: "127.0.0.1", internal: true } as never] }), false);
+ assert.equal(interfacePresent("wg0", { wg0: [{ address: "10.2.0.2", internal: false } as never] }), true);
+});
+
+test("unconfigured, it refuses to start", async () => {
+ await assert.rejects(
+ runSeeder({ paths: {} as Paths, settings: DEFAULT_SEEDER_SETTINGS, onLog: () => {} }),
+ /not configured/,
+ );
+});
+
+test("the tracker tracks only the torrents it is told to", { timeout: 30_000 }, async () => {
+ const tracker = await startTracker({ host: "127.0.0.1", port: 0, allowed: () => new Set(["a".repeat(40)]) });
+ try {
+ // An announce for a torrent it was not told about: the 20-byte infohash
+ // percent-encoded byte by byte, as a BitTorrent client sends it.
+ const raw = Buffer.from("b".repeat(40), "hex");
+ const infoHash = [...raw].map((b) => `%${b.toString(16).padStart(2, "0")}`).join("");
+ const q = `info_hash=${infoHash}&peer_id=${"-TT0000-".padEnd(20, "0")}&port=6881&uploaded=0&downloaded=0&left=0&compact=1`;
+ const body = await (await fetch(`http://127.0.0.1:${tracker.port}/announce?${q}`)).text();
+ assert.match(body, /failure reason.*does not track/);
+ assert.equal(tracker.swarm("b".repeat(40)), null);
+ } finally {
+ await tracker.close();
+ }
+});
+
+test("TCP: it seeds, a leecher completes, the seeder stands by while the leecher seeds and resumes when it leaves", { timeout: 60_000 }, async () => {
+ const f = await fixture((port) => `http://127.0.0.1:${port}/announce`);
+ const ac = new AbortController();
+ const transitions: string[] = [];
+ const log: string[] = [];
+ const seederClient = newClient();
+ const peer = newClient();
+ const running = runSeeder({
+ paths: {} as Paths,
+ settings: settings(),
+ onLog: (l) => log.push(l),
+ signal: ac.signal,
+ deps: {
+ targets: async () => [f.target],
+ pollMs: 300,
+ policy: { standbyAfterSeconds: 1, resumeAfterSeconds: 1 },
+ onTransition: (t) => transitions.push(t.to),
+ client: seederClient,
+ },
+ });
+ try {
+ const out = path.join(f.dir, "leech");
+ await mkdir(out);
+ await leechTo(peer, f.made.torrent, out, 30_000, log).done;
+ assert.deepEqual(await readFile(path.join(out, `${ID}.mp4`)), f.bytes);
+ // The leecher now seeds too: the home seeder steps back.
+ await until(() => transitions.includes("standby"), 20_000, `standby; log: ${log.join(" | ")}`);
+ assert.match(log.join("\n"), /seeding → standby — 1 other seeder\(s\) on every poll/);
+ // The leecher leaves: nobody else seeds, so it comes back.
+ await destroy(peer);
+ await until(() => transitions.at(-1) === "seeding", 20_000, `resume; log: ${log.join(" | ")}`);
+ assert.match(log.join("\n"), /standby → seeding — no other seeder/);
+ } finally {
+ ac.abort();
+ await running.catch(() => {});
+ await destroy(peer);
+ await destroy(seederClient);
+ await f.tracker.close();
+ await rm(f.dir, { recursive: true, force: true });
+ }
+});
+
+test("WebRTC: a peer reaching it through the WebSocket tracker completes (node-datachannel)", { timeout: 60_000 }, async (t) => {
+ if (!WebTorrent.WEBRTC_SUPPORT) {
+ t.skip("no WebRTC in this node");
+ return;
+ }
+ const f = await fixture((port) => `ws://127.0.0.1:${port}`);
+ const ac = new AbortController();
+ const log: string[] = [];
+ const seederClient = newClient();
+ const peer = newClient();
+ const running = runSeeder({
+ paths: {} as Paths,
+ settings: settings(),
+ onLog: (l) => log.push(l),
+ signal: ac.signal,
+ deps: {
+ targets: async () => [f.target],
+ pollMs: 1000,
+ policy: { standbyAfterSeconds: 3600, resumeAfterSeconds: 3600 },
+ client: seederClient,
+ },
+ });
+ try {
+ // The leecher announces AFTER the seeder's announce is in. Two peers whose
+ // WebSocket offers cross in flight each answer the other's and each keep
+ // the connection the other dropped — an artefact of starting both in the
+ // same instant, which a browser arriving at a running seeder does not do.
+ await until(() => (f.tracker.swarm(f.made.infoHash)?.complete ?? 0) >= 1, 20_000, "the seeder's announce");
+ const out = path.join(f.dir, "leech");
+ await mkdir(out);
+ const { t: tor, done } = leechTo(peer, f.made.torrent, out, 40_000, log);
+ await done;
+ assert.deepEqual(await readFile(path.join(out, `${ID}.mp4`)), f.bytes);
+ assert.ok(tor.wires.some((w) => w.type === "webrtc"), `wires: ${tor.wires.map((w) => w.type).join(",")}`);
+ } finally {
+ ac.abort();
+ await running.catch(() => {});
+ await destroy(peer);
+ await destroy(seederClient);
+ await f.tracker.close();
+ await rm(f.dir, { recursive: true, force: true });
+ }
+});
diff --git a/common/controller/seeder.ts b/common/controller/seeder.ts
@@ -0,0 +1,372 @@
+// `archilyzer seed` — the home seeder of last resort (release 21 D4b), and
+// `archilyzer tracker`, the pilot's self-hosted tracker.
+//
+// THE SEEDER. WebTorrent 3 in node: TCP for desktop clients, WebRTC (through
+// node-datachannel, which WebTorrent's peer library loads) for browsers. It
+// seeds the playable torrents (controller/preparePlayable.ts) of every channel
+// of the sites named in `settings.seeder.sites` — and, per torrent, only while
+// no other seeder has it: lib/lastResortSeeder.ts decides, this module
+// observes (each tracker's scrape, its own wires) and applies. STANDBY is the
+// torrent removed from the client with its data kept (`stopped` is announced,
+// every peer closed); SEEDING is it added back, trusting the copy on disk
+// (`skipVerify`: the manifest's size was checked, and every peer verifies each
+// piece it takes).
+//
+// No DHT, no local discovery, no UPnP/NAT-PMP, no web seeds: the seeder is
+// found through the trackers it is given and nowhere else, and it opens no
+// port on a router. `bindInterface`, when set, must exist (the VPN tunnel):
+// the seeder waits for it to start, and drops every torrent the moment it is
+// gone — the docker profile's namespace is the real kill switch
+// (docker-compose.seeder.yml); this is the same rule in the app.
+//
+// THE TRACKER. bittorrent-tracker's server, HTTP + WebSocket (no UDP), bound
+// to the address given (127.0.0.1 by default), tracking ONLY the infohashes
+// in the playable manifests of the seeder's sites — an open tracker on a home
+// connection would be anyone's.
+//
+// SERVER-ONLY.
+
+/// <reference path="../types/webtorrent-modules.d.ts" />
+import os from "node:os";
+import path from "node:path";
+import { readFile, stat } from "node:fs/promises";
+import WebTorrent, { type Torrent } from "webtorrent";
+import { Client as TrackerClient, Server as TrackerServer, type ScrapeData } from "bittorrent-tracker";
+import parseTorrent from "parse-torrent";
+import type { Paths } from "../lib/paths";
+import { getSite, siteChannelSlugs } from "../lib/site";
+import { readChannelConfig } from "./channels";
+import { defaultPlayableRoot, loadPlayableManifest } from "./preparePlayable";
+import {
+ initialSeederMemory,
+ stepSeeder,
+ type SeederMemory,
+ type SeederPolicy,
+ type SwarmObservation,
+} from "../lib/lastResortSeeder";
+import { seederConfigured, type SeederSettings } from "../lib/seederSettings";
+
+export type SeedTarget = {
+ slug: string;
+ id: string;
+ infoHash: string;
+ torrentPath: string;
+ // The torrent's one file (`<id>.<ext>`), and the directory holding it.
+ file: string;
+ dir: string;
+ bytes: number;
+};
+
+// Every playable torrent of the sites' channels, deduplicated by infohash.
+// A site that does not load, or a channel with no manifest, contributes none.
+export async function listSeedTargets(
+ paths: Paths,
+ settings: Pick<SeederSettings, "sites">,
+ onLog: (line: string) => void = () => {},
+): Promise<SeedTarget[]> {
+ const out = new Map<string, SeedTarget>();
+ const slugs = new Set<string>();
+ for (const siteId of settings.sites) {
+ try {
+ for (const slug of siteChannelSlugs(getSite(siteId, paths))) slugs.add(slug);
+ } catch (err) {
+ onLog(`site ${siteId}: ${(err as Error).message} — skipped`);
+ }
+ }
+ for (const slug of [...slugs].sort()) {
+ const config = await readChannelConfig(paths, slug).catch(() => null);
+ const root = await defaultPlayableRoot(paths, config ?? null);
+ const manifest = await loadPlayableManifest(root, slug);
+ for (const [id, e] of Object.entries(manifest.entries)) {
+ if (out.has(e.infoHash)) continue;
+ out.set(e.infoHash, {
+ slug,
+ id,
+ infoHash: e.infoHash,
+ torrentPath: path.join(root, slug, e.torrent),
+ file: path.join(root, slug, e.file),
+ dir: path.dirname(path.join(root, slug, e.file)),
+ bytes: e.bytes,
+ });
+ }
+ }
+ return [...out.values()];
+}
+
+export function interfacePresent(
+ name: string,
+ interfaces: NodeJS.Dict<os.NetworkInterfaceInfo[]> = os.networkInterfaces(),
+): boolean {
+ return (interfaces[name] ?? []).some((a) => !a.internal);
+}
+
+export type ScrapeFn = (
+ announce: string,
+ infoHashes: string[],
+) => Promise<Map<string, { complete: number; incomplete: number }>>;
+
+// One scrape request per tracker, for every infohash at once.
+export const scrapeTracker: ScrapeFn = (announce, infoHashes) =>
+ new Promise((resolve, reject) => {
+ if (infoHashes.length === 0) return resolve(new Map());
+ TrackerClient.scrape({ announce, infoHash: infoHashes }, (err, data) => {
+ if (err) return reject(err);
+ const rows: ScrapeData[] =
+ infoHashes.length === 1 ? [data as ScrapeData] : Object.values(data as Record<string, ScrapeData>);
+ resolve(new Map(rows.map((r) => [r.infoHash, { complete: r.complete, incomplete: r.incomplete }])));
+ });
+ });
+
+// Several trackers' answers for one torrent → one: the most seeders and the
+// most leechers any tracker reported (the swarms overlap; a sum would count
+// one browser on two trackers twice). Null when no tracker answered.
+export function mergeScrapes(
+ answers: (Map<string, { complete: number; incomplete: number }> | null)[],
+ infoHash: string,
+): { complete: number; incomplete: number } | null {
+ let out: { complete: number; incomplete: number } | null = null;
+ for (const a of answers) {
+ const row = a?.get(infoHash);
+ if (!row) continue;
+ out = out
+ ? { complete: Math.max(out.complete, row.complete), incomplete: Math.max(out.incomplete, row.incomplete) }
+ : { ...row };
+ }
+ return out;
+}
+
+export type RunSeederOptions = {
+ paths: Paths;
+ settings: SeederSettings;
+ onLog: (line: string) => void;
+ signal?: AbortSignal;
+ deps?: {
+ now?: () => number;
+ sleep?: (ms: number, signal?: AbortSignal) => Promise<void>;
+ scrape?: ScrapeFn;
+ interfaces?: () => NodeJS.Dict<os.NetworkInterfaceInfo[]>;
+ targets?: () => Promise<SeedTarget[]>;
+ // Overrides for a test's clock: the poll and the policy's windows, in ms.
+ pollMs?: number;
+ policy?: SeederPolicy;
+ // Stop after this many polls (tests).
+ maxPolls?: number;
+ onTransition?: (t: { infoHash: string; to: string; reason: string }) => void;
+ client?: WebTorrent;
+ };
+};
+
+const defaultSleep = (ms: number, signal?: AbortSignal) =>
+ new Promise<void>((resolve) => {
+ const t = setTimeout(resolve, ms);
+ signal?.addEventListener("abort", () => {
+ clearTimeout(t);
+ resolve();
+ }, { once: true });
+ });
+
+function addTorrent(client: WebTorrent, buf: Buffer, target: SeedTarget, trackers: string[]): Promise<Torrent> {
+ return new Promise((resolve, reject) => {
+ const t = client.add(
+ buf,
+ { path: target.dir, skipVerify: true, ...(trackers.length ? { announce: trackers } : {}) },
+ (torrent) => resolve(torrent),
+ );
+ t.once("error", reject);
+ });
+}
+
+function removeTorrent(client: WebTorrent, torrent: Torrent): Promise<void> {
+ return new Promise((resolve) => {
+ client.remove(torrent, { destroyStore: false }, () => resolve()).catch(() => resolve());
+ });
+}
+
+export async function runSeeder(opts: RunSeederOptions): Promise<void> {
+ const { settings, onLog } = opts;
+ const d = opts.deps ?? {};
+ const now = d.now ?? Date.now;
+ const sleep = d.sleep ?? defaultSleep;
+ const scrape = d.scrape ?? scrapeTracker;
+ const ifaces = d.interfaces ?? (() => os.networkInterfaces());
+ const pollMs = d.pollMs ?? settings.pollSeconds * 1000;
+ const policy: SeederPolicy = d.policy ?? {
+ standbyAfterSeconds: settings.standbyAfterSeconds,
+ resumeAfterSeconds: settings.standbyAfterSeconds,
+ };
+ if (!seederConfigured(settings)) {
+ throw new Error("the seeder is not configured: settings.seeder.sites is empty");
+ }
+ const client =
+ d.client ??
+ new WebTorrent({
+ maxConns: settings.maxConnections,
+ uploadLimit: settings.maxUploadKiBps > 0 ? settings.maxUploadKiBps * 1024 : -1,
+ dht: false,
+ lsd: false,
+ natUpnp: false,
+ natPmp: false,
+ utp: false,
+ webSeeds: false,
+ });
+ onLog(`seeder: WebRTC ${WebTorrent.WEBRTC_SUPPORT ? "available" : "NOT available (TCP peers only)"}; sites ${settings.sites.join(", ")}`);
+
+ type Entry = { target: SeedTarget; memory: SeederMemory; torrent: Torrent | null; trackers: string[]; buf: Buffer };
+ const entries = new Map<string, Entry>();
+ let tunnelUp = true;
+ let polls = 0;
+
+ try {
+ while (!opts.signal?.aborted) {
+ polls += 1;
+ // The tunnel, first: no tunnel, no torrents.
+ if (settings.bindInterface) {
+ const up = interfacePresent(settings.bindInterface, ifaces());
+ if (!up && tunnelUp) {
+ onLog(`seeder: ${settings.bindInterface} is gone — every torrent stopped until it is back`);
+ for (const e of entries.values()) {
+ if (e.torrent) await removeTorrent(client, e.torrent);
+ e.torrent = null;
+ }
+ } else if (up && !tunnelUp) {
+ onLog(`seeder: ${settings.bindInterface} is back`);
+ }
+ tunnelUp = up;
+ }
+
+ if (tunnelUp) {
+ // The targets, re-read each poll: a new prepare adds torrents.
+ const targets = await (d.targets ?? (() => listSeedTargets(opts.paths, settings, onLog)))();
+ const wanted = new Set(targets.map((t) => t.infoHash));
+ for (const [hash, e] of entries) {
+ if (!wanted.has(hash)) {
+ if (e.torrent) await removeTorrent(client, e.torrent);
+ entries.delete(hash);
+ onLog(`${e.target.id}: no longer in a playable manifest — dropped`);
+ }
+ }
+ for (const target of targets) {
+ if (entries.has(target.infoHash)) continue;
+ // skipVerify trusts the copy, so the copy must at least be whole.
+ const st = await stat(target.file).catch(() => null);
+ if (!st || st.size !== target.bytes) {
+ onLog(`${target.id}: ${target.file} is ${st ? `${st.size} bytes, not ${target.bytes}` : "not there"} — skipped`);
+ continue;
+ }
+ const buf = await readFile(target.torrentPath).catch(() => null);
+ if (!buf) {
+ onLog(`${target.id}: ${target.torrentPath} is not there — skipped`);
+ continue;
+ }
+ const own = (await parseTorrent(buf)).announce ?? [];
+ const trackers = settings.trackers.length ? settings.trackers : own;
+ entries.set(target.infoHash, { target, memory: initialSeederMemory(now()), torrent: null, trackers, buf });
+ }
+ // Every torrent the machine says to seed and is not seeding, added.
+ for (const e of entries.values()) {
+ if (e.memory.state === "seeding" && !e.torrent) {
+ try {
+ e.torrent = await addTorrent(client, e.buf, e.target, e.trackers);
+ onLog(`${e.target.id}: seeding (${e.target.infoHash})`);
+ } catch (err) {
+ onLog(`${e.target.id}: could not seed — ${(err as Error).message}`);
+ }
+ }
+ }
+
+ // Observe and decide.
+ const byTracker = new Map<string, string[]>();
+ for (const e of entries.values()) {
+ for (const tr of e.trackers) byTracker.set(tr, [...(byTracker.get(tr) ?? []), e.target.infoHash]);
+ }
+ const answers = new Map<string, Map<string, { complete: number; incomplete: number }> | null>();
+ for (const [tr, hashes] of byTracker) {
+ answers.set(tr, await scrape(tr, hashes).catch((err: Error) => {
+ onLog(`scrape ${tr}: ${err.message}`);
+ return null;
+ }));
+ }
+ for (const e of entries.values()) {
+ const wires = e.torrent?.wires ?? [];
+ // By peer, not by wire: two peers that dialled each other at once
+ // hold two wires.
+ const peers = (seeds: boolean) =>
+ new Set(wires.filter((w) => w.isSeeder === seeds).map((w, i) => w.peerId ?? w.remoteAddress ?? `wire${i}`)).size;
+ const o: SwarmObservation = {
+ at: now(),
+ scrape: mergeScrapes(e.trackers.map((tr) => answers.get(tr) ?? null), e.target.infoHash),
+ selfCounted: e.memory.state === "seeding" && !!e.torrent,
+ connectedSeeds: peers(true),
+ connectedLeechers: peers(false),
+ };
+ const step = stepSeeder(e.memory, o, policy);
+ e.memory = step.memory;
+ if (!step.transition) continue;
+ onLog(`${e.target.id}: ${step.transition.from} → ${step.transition.to} — ${step.transition.reason} (${step.seen})`);
+ d.onTransition?.({ infoHash: e.target.infoHash, to: step.transition.to, reason: step.transition.reason });
+ if (step.transition.to === "standby" && e.torrent) {
+ await removeTorrent(client, e.torrent);
+ e.torrent = null;
+ } else if (step.transition.to === "seeding" && !e.torrent) {
+ try {
+ e.torrent = await addTorrent(client, e.buf, e.target, e.trackers);
+ } catch (err) {
+ onLog(`${e.target.id}: could not resume — ${(err as Error).message}`);
+ }
+ }
+ }
+ }
+ if (d.maxPolls && polls >= d.maxPolls) break;
+ await sleep(pollMs, opts.signal);
+ }
+ } finally {
+ if (!d.client) await new Promise<void>((resolve) => client.destroy(() => resolve()));
+ }
+}
+
+// --- the tracker ------------------------------------------------------------
+
+export type RunningTracker = {
+ port: number;
+ host: string;
+ // What the tracker itself knows of one swarm (null: never announced).
+ swarm: (infoHash: string) => { complete: number; incomplete: number } | null;
+ close: () => Promise<void>;
+};
+
+export async function startTracker(opts: {
+ host: string;
+ port: number;
+ // The infohashes it tracks; anything else is refused. Re-read on every
+ // announce through this function, so a new prepare needs no restart.
+ allowed: () => Promise<Set<string>> | Set<string>;
+ onLog?: (line: string) => void;
+}): Promise<RunningTracker> {
+ const server = new TrackerServer({
+ udp: false,
+ http: true,
+ ws: true,
+ stats: false,
+ trustProxy: false,
+ filter: (infoHash, _params, cb) => {
+ Promise.resolve(opts.allowed())
+ .then((set) => cb(set.has(infoHash) ? null : new Error("this tracker does not track that torrent")))
+ .catch((err: Error) => cb(err));
+ },
+ });
+ server.on("error", (err: Error) => opts.onLog?.(`tracker: ${err.message}`));
+ server.on("warning", (err: Error) => opts.onLog?.(`tracker warning: ${err.message}`));
+ await new Promise<void>((resolve) => server.listen(opts.port, opts.host, () => resolve()));
+ const address = server.http?.address();
+ const port = address?.port ?? opts.port;
+ opts.onLog?.(`tracker: http://${opts.host}:${port}/announce and ws://${opts.host}:${port}`);
+ return {
+ port,
+ host: opts.host,
+ swarm: (infoHash) => {
+ const t = server.torrents[infoHash];
+ return t ? { complete: t.complete, incomplete: t.incomplete } : null;
+ },
+ close: () => new Promise<void>((resolve) => server.close(() => resolve())),
+ };
+}
diff --git a/common/jobs/jobKinds.test.ts b/common/jobs/jobKinds.test.ts
@@ -293,3 +293,29 @@ test("release 18: every drainable kind but the lane runners and the clip-window
}
assert.equal(isIngestKind("no-such-kind"), false);
});
+
+// RELEASE 21 D1: local media attached to held videos. It writes the big file
+// (the store), stops between videos, and its pointers and created records are
+// what the index reads — so media, drainable, and ingest.
+test("release 21: attach-media is a drainable media ingest kind, not replayable", () => {
+ const meta = getJobKind("attach-media");
+ assert.ok(meta, "registered");
+ assert.equal(jobKindLabel("attach-media"), "Attach local media");
+ assert.equal(isDrainableKind("attach-media"), true);
+ assert.equal(meta.replayable, false);
+ assert.equal(kindNeedsMedia("attach-media"), true);
+ assert.equal(kindNeedsText("attach-media"), false);
+ assert.equal(isIngestKind("attach-media"), true);
+});
+
+// RELEASE 21 D3: playable copies and torrents. Reads and writes big files,
+// stops between videos, and writes the manifest a site's build reads.
+test("release 21: prepare-playable is a drainable media ingest kind, not replayable", () => {
+ const meta = getJobKind("prepare-playable");
+ assert.ok(meta, "registered");
+ assert.equal(jobKindLabel("prepare-playable"), "Prepare playable copies");
+ assert.equal(isDrainableKind("prepare-playable"), true);
+ assert.equal(meta.replayable, false);
+ assert.equal(kindNeedsMedia("prepare-playable"), true);
+ assert.equal(isIngestKind("prepare-playable"), true);
+});
diff --git a/common/jobs/jobKinds.ts b/common/jobs/jobKinds.ts
@@ -962,6 +962,40 @@ const JOB_KINDS: Record<string, JobKindMeta> = {
queueKeyStrategy: "custom",
needsMedia: true,
},
+ // LOCAL MEDIA ATTACHED TO HELD VIDEOS (release 21 D1,
+ // controller/attachMedia.ts): each video's file copied out of a local
+ // archive (a directory, a zip read in place, a 7z) into the saved-video
+ // store. Media: it writes the big file. Drainable: it stops between videos,
+ // and a re-run resumes. An INGEST kind: it writes `saved-video.json` beside
+ // each record and, with `createRecords`, whole new records the index reads.
+ // On the channel's own queue (`channel:<slug>`): nothing is fetched, so no
+ // platform queue is owed a turn. Not replayable: the archive it names is a
+ // path on some drive, and a retry is a re-run.
+ "attach-media": {
+ kind: "attach-media",
+ label: "Attach local media",
+ drainable: true,
+ replayable: false,
+ queueKeyStrategy: "custom",
+ needsMedia: true,
+ },
+ // PLAYABLE COPIES AND THEIR TORRENTS (release 21 D3,
+ // controller/preparePlayable.ts): each saved container remuxed losslessly
+ // into a browser's container, and one torrent per copy. Media: it reads the
+ // saved container and writes copies as big. Drainable: it stops between
+ // videos. An INGEST kind: the playable manifest it writes is what a site's
+ // build reads for its torrents (D4). One queue machine-wide (`playable`):
+ // a remux is disk-bound on the archive drive, and two at once only thrash
+ // it. Heavy: each remux takes release 19's heavy slot (`pnpm heavy`'s gate),
+ // one video at a time, so builds and e2e interleave with a long run.
+ "prepare-playable": {
+ kind: "prepare-playable",
+ label: "Prepare playable copies",
+ drainable: true,
+ replayable: false,
+ queueKeyStrategy: "custom",
+ needsMedia: true,
+ },
};
export function getJobKind(kind: string): JobKindMeta | undefined {
@@ -1026,6 +1060,8 @@ const INGEST_KINDS: ReadonlySet<string> = new Set([
"check-availability",
"quick-availability-check",
"check-maybe-missing",
+ "attach-media",
+ "prepare-playable",
]);
export function isIngestKind(kind: string | undefined): boolean {
diff --git a/common/lib/attachMediaPlan.test.ts b/common/lib/attachMediaPlan.test.ts
@@ -0,0 +1,178 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import {
+ archiveFolderInfoJson,
+ countAttachRows,
+ descriptionFromText,
+ groupArchiveFolders,
+ idFromFolderName,
+ idFromMediaName,
+ planAttach,
+ publishedDateFromText,
+ titleFromFolderName,
+ type SourceFile,
+} from "./attachMediaPlan";
+
+// Run with: pnpm --filter yt-dlp-transcript-common exec tsx --test lib/attachMediaPlan.test.ts
+//
+// The folder shapes are the pilot archive's (release 21), with synthetic ids
+// and titles: `NNN - <title> - (<id>)` holding either yt-dlp's files
+// (`<title>-<id>.<ext>` + `.info.json`) or a bare `<id>.<ext>` beside
+// description.txt / source.txt, and `[LOST]` folders with no media.
+
+const f = (p: string, size = 10): SourceFile => ({ path: p, size });
+
+const ARCHIVE: SourceFile[] = [
+ f("YouTube/README.txt"),
+ f("YouTube/001 - Intro - (aaaaaaaaaaa)/Intro-aaaaaaaaaaa.mkv", 1000),
+ f("YouTube/001 - Intro - (aaaaaaaaaaa)/Intro-aaaaaaaaaaa.info.json"),
+ f("YouTube/001 - Intro - (aaaaaaaaaaa)/Intro-aaaaaaaaaaa.jpg"),
+ f("YouTube/002 [LOST] - Gone - (bbbbbbbbbbb)/description.txt"),
+ f("YouTube/002 [LOST] - Gone - (bbbbbbbbbbb)/source.txt"),
+ f("YouTube/003 - Episode 2 - (ccccccccccc)/ccccccccccc.mp4", 2000),
+ f("YouTube/003 - Episode 2 - (ccccccccccc)/description.txt"),
+ f("YouTube/003 - Episode 2 - (ccccccccccc)/Sources.txt"),
+ f("YouTube/004 - Text only - (ddddddddddd)/description.txt"),
+ f("YouTube/005 - Two files - (eeeeeeeeeee)/a.mp4"),
+ f("YouTube/005 - Two files - (eeeeeeeeeee)/b.webm"),
+ f("YouTube/006 - Odd name - (fffffffffff)/Odd name [watch-fffffffffff-1].mp4", 3000),
+ f("loose/Some title [ggggggggggg].webm", 4000),
+ f("loose/no id here.mp4"),
+];
+
+test("ids: the folder's trailing (id), else the file's yt-dlp suffix", () => {
+ assert.equal(idFromFolderName("001 - Intro - (aaaaaaaaaaa)"), "aaaaaaaaaaa");
+ assert.equal(idFromFolderName("Intro (too-short)"), null);
+ assert.equal(idFromMediaName("x/Intro-aaaaaaaaaaa.mkv"), "aaaaaaaaaaa");
+ assert.equal(idFromMediaName("ccccccccccc.mp4"), "ccccccccccc");
+ assert.equal(idFromMediaName("Some title [ggggggggggg].webm"), "ggggggggggg");
+ assert.equal(idFromMediaName("no id here.mp4"), null);
+});
+
+test("folders: grouped by parent, sidecars found, [LOST] marked", () => {
+ const folders = groupArchiveFolders(ARCHIVE);
+ const by = (id: string) => folders.find((x) => x.id === id)!;
+ assert.equal(by("aaaaaaaaaaa").idFrom, "folder");
+ assert.equal(by("aaaaaaaaaaa").media.length, 1);
+ assert.ok(by("aaaaaaaaaaa").infoJson);
+ assert.equal(by("bbbbbbbbbbb").lost, true);
+ assert.ok(by("ccccccccccc").descriptionTxt);
+ assert.ok(by("ccccccccccc").sourceTxt, "Sources.txt, any case");
+ assert.equal(by("ggggggggggg").idFrom, "file");
+ // The archive's root holds no media and yields a folder with no id.
+ assert.ok(folders.some((x) => x.folder === "YouTube" && x.id === null && x.media.length === 0));
+ // A folder with no id splits per media file.
+ assert.ok(folders.some((x) => x.folder === "loose" && x.id === null && x.media.length === 1));
+});
+
+test("plan: every class, and held videos the archive has no media for", () => {
+ const folders = groupArchiveFolders(ARCHIVE);
+ const plan = planAttach({
+ folders,
+ held: new Set(["aaaaaaaaaaa", "bbbbbbbbbbb", "fffffffffff", "hhhhhhhhhhh", "ddddddddddd"]),
+ attached: new Set(["fffffffffff"]),
+ });
+ const cls = Object.fromEntries(plan.rows.map((r) => [r.id ?? r.folder, r.class]));
+ assert.equal(cls.aaaaaaaaaaa, "attach");
+ assert.equal(cls.bbbbbbbbbbb, "lost");
+ assert.equal(cls.ccccccccccc, "not-held");
+ assert.equal(cls.ddddddddddd, "no-media");
+ assert.equal(cls.eeeeeeeeeee, "ambiguous");
+ assert.equal(cls.fffffffffff, "already-attached");
+ assert.equal(cls.ggggggggggg, "not-held");
+ assert.equal(cls.loose, "unmatched");
+ assert.deepEqual(plan.heldWithoutMedia, ["bbbbbbbbbbb", "ddddddddddd", "hhhhhhhhhhh"]);
+ const counts = countAttachRows(plan.rows);
+ assert.equal(counts.attach, 1);
+ assert.equal(counts["not-held"], 2);
+});
+
+test("plan: createRecords turns not-held into create; replace re-attaches", () => {
+ const folders = groupArchiveFolders(ARCHIVE);
+ const plan = planAttach({
+ folders,
+ held: new Set(["fffffffffff"]),
+ attached: new Set(["fffffffffff"]),
+ createRecords: true,
+ replace: true,
+ });
+ const row = (id: string) => plan.rows.find((r) => r.id === id)!;
+ assert.equal(row("ccccccccccc").class, "create");
+ assert.ok(row("ccccccccccc").source?.descriptionTxt, "the folder rides along for the record");
+ assert.equal(row("fffffffffff").class, "attach");
+ assert.equal(row("fffffffffff").replacing, true);
+});
+
+test("plan: two folders claiming one id are both ambiguous", () => {
+ const folders = groupArchiveFolders([
+ f("a/001 - X - (aaaaaaaaaaa)/x.mp4"),
+ f("b/001 - X again - (aaaaaaaaaaa)/y.mp4"),
+ ]);
+ const plan = planAttach({ folders, held: new Set(["aaaaaaaaaaa"]), attached: new Set() });
+ assert.deepEqual(plan.rows.map((r) => r.class), ["ambiguous", "ambiguous"]);
+ assert.deepEqual(plan.heldWithoutMedia, [], "an ambiguous id is not 'no media'");
+});
+
+test("plan: match narrows the folders and drops the held-without-media list", () => {
+ const plan = planAttach({
+ folders: groupArchiveFolders(ARCHIVE),
+ held: new Set(["aaaaaaaaaaa", "ccccccccccc"]),
+ attached: new Set(),
+ match: /episode 2/i,
+ });
+ assert.deepEqual(plan.rows.map((r) => [r.id, r.class]), [["ccccccccccc", "attach"]]);
+ assert.equal(plan.heldWithoutMedia, null);
+});
+
+test("plan: explicit items are the whole plan, by path", () => {
+ const plan = planAttach({
+ folders: groupArchiveFolders(ARCHIVE),
+ held: new Set(["zzzzzzzzzzz"]),
+ attached: new Set(),
+ files: ARCHIVE,
+ items: [
+ { id: "zzzzzzzzzzz", path: "YouTube/005 - Two files - (eeeeeeeeeee)/b.webm" },
+ { id: "yyyyyyyyyyy", path: "nope.mp4" },
+ ],
+ });
+ assert.deepEqual(plan.rows.map((r) => [r.id, r.class, r.media?.path ?? null]), [
+ ["zzzzzzzzzzz", "attach", "YouTube/005 - Two files - (eeeeeeeeeee)/b.webm"],
+ ["yyyyyyyyyyy", "no-media", null],
+ ]);
+});
+
+test("records: title from the folder name, date and description from description.txt", () => {
+ assert.equal(titleFromFolderName("006 - The Show Episode 002 - (zz1Yy2Xx3Ww)"), "The Show Episode 002");
+ assert.equal(titleFromFolderName("002 [LOST] - Gone - (bbbbbbbbbbb)"), "Gone");
+ assert.equal(titleFromFolderName("News, Weather & More! - (q_9Rr8Ss7Tt)"), "News, Weather & More!");
+ assert.equal(publishedDateFromText("Published on July 13th 2018\n\nTips"), "20180713");
+ assert.equal(publishedDateFromText("Published on March 1, 2019"), "20190301");
+ assert.equal(publishedDateFromText("no date"), null);
+ assert.equal(descriptionFromText("Published on July 13th 2018\n\nTips & comments\nCategory: X"), "Tips & comments\nCategory: X");
+
+ const info = archiveFolderInfoJson({
+ id: "ccccccccccc",
+ folderName: "003 - Episode 2 - (ccccccccccc)",
+ descriptionText: "Published on July 13th 2018\n\nTips",
+ platform: "youtube",
+ durationSec: 61.234,
+ });
+ assert.deepEqual(info, {
+ id: "ccccccccccc",
+ title: "Episode 2",
+ description: "Tips",
+ duration: 61.23,
+ webpage_url: "https://www.youtube.com/watch?v=ccccccccccc",
+ original_url: "https://www.youtube.com/watch?v=ccccccccccc",
+ extractor: "youtube",
+ extractor_key: "Youtube",
+ upload_date: "20180713",
+ });
+ const keys = Object.keys(info);
+ assert.equal(keys[1], "title", "title near the head");
+ assert.equal(keys.at(-1), "upload_date", "upload_date last, in the tail");
+ // Another platform: no invented YouTube URL.
+ const other = archiveFolderInfoJson({ id: "x", folderName: "x", platform: "rumble" });
+ assert.equal(other.webpage_url, undefined);
+ assert.equal(other.extractor, undefined);
+});
diff --git a/common/lib/attachMediaPlan.ts b/common/lib/attachMediaPlan.ts
@@ -0,0 +1,343 @@
+// WHICH FILE OF A LOCAL ARCHIVE IS WHICH VIDEO (release 21 D1).
+//
+// The pure half of `attach-media` (controller/attachMedia.ts): given the file
+// list of an archive (a zip's entries, a directory's files), group it into
+// per-video folders, find each folder's video id and media file, and classify
+// every folder against what the channel holds. No fs: the controller lists,
+// this decides, and the tests run over plain arrays.
+//
+// THE ID, in order: the folder's trailing `(<11-char id>)` — the archive's own
+// naming, `NNN - <title> - (<id>)` — else the media file's yt-dlp suffix
+// (`<title>-<id>.<ext>`, `<title> [<id>].<ext>`, or a bare `<id>.<ext>`), else
+// an explicit item. The 11-character shape is YouTube's; any other platform's
+// files are attached by explicit `items`.
+//
+// `[LOST]` folders (the archive's mark for a video whose media nobody has)
+// are listed and never attached, whatever they hold.
+
+export const ATTACHABLE_MEDIA_EXTS: readonly string[] = ["mp4", "mkv", "webm", "mov", "m4v"];
+
+const YT_ID = "[A-Za-z0-9_-]{11}";
+const FOLDER_ID_RE = new RegExp(`\\((${YT_ID})\\)\\s*$`);
+const FILE_ID_RES = [
+ new RegExp(`^(${YT_ID})\\.[A-Za-z0-9]+$`),
+ new RegExp(`-(${YT_ID})\\.[A-Za-z0-9]+$`),
+ new RegExp(`\\[(${YT_ID})\\]\\.[A-Za-z0-9]+$`),
+];
+
+export type SourceFile = {
+ // Path inside the archive, "/"-separated, no leading "/".
+ path: string;
+ size: number;
+};
+
+export type ArchiveFolder = {
+ // The folder's path inside the archive ("" for the archive's root).
+ folder: string;
+ // Its last segment (the root's is "").
+ name: string;
+ id: string | null;
+ idFrom: "folder" | "file" | null;
+ lost: boolean;
+ // Every video container directly in the folder.
+ media: SourceFile[];
+ // The sidecars createRecords reads, when present.
+ infoJson?: SourceFile;
+ descriptionTxt?: SourceFile;
+ sourceTxt?: SourceFile;
+};
+
+function extOf(name: string): string {
+ const dot = name.lastIndexOf(".");
+ return dot < 0 ? "" : name.slice(dot + 1).toLowerCase();
+}
+
+function baseName(p: string): string {
+ const i = p.lastIndexOf("/");
+ return i < 0 ? p : p.slice(i + 1);
+}
+
+function dirName(p: string): string {
+ const i = p.lastIndexOf("/");
+ return i < 0 ? "" : p.slice(0, i);
+}
+
+export function isAttachableMedia(name: string): boolean {
+ return ATTACHABLE_MEDIA_EXTS.includes(extOf(name));
+}
+
+export function idFromFolderName(name: string): string | null {
+ return FOLDER_ID_RE.exec(name)?.[1] ?? null;
+}
+
+export function idFromMediaName(name: string): string | null {
+ const base = baseName(name);
+ for (const re of FILE_ID_RES) {
+ const m = re.exec(base);
+ if (m) return m[1];
+ }
+ return null;
+}
+
+export function isLostFolder(name: string): boolean {
+ return /\[LOST\]/i.test(name);
+}
+
+// One folder per directory that holds a file. A media file's folder is its
+// immediate parent; the id comes from that folder's name, else the file's.
+export function groupArchiveFolders(files: readonly SourceFile[]): ArchiveFolder[] {
+ const byFolder = new Map<string, SourceFile[]>();
+ for (const f of files) {
+ if (!f.path || f.path.endsWith("/")) continue;
+ const folder = dirName(f.path);
+ const list = byFolder.get(folder) ?? [];
+ list.push(f);
+ byFolder.set(folder, list);
+ }
+ const out: ArchiveFolder[] = [];
+ for (const [folder, list] of [...byFolder].sort(([a], [b]) => a.localeCompare(b))) {
+ const name = baseName(folder);
+ const media = list.filter((f) => isAttachableMedia(f.path));
+ const pick = (test: (base: string) => boolean) => list.find((f) => test(baseName(f.path)));
+ const infoJson = pick((b) => b.toLowerCase().endsWith(".info.json"));
+ const descriptionTxt = pick((b) => b.toLowerCase() === "description.txt");
+ const sourceTxt = pick((b) => /^sources?\.txt$/i.test(b));
+ const lost = isLostFolder(name);
+ const base = {
+ lost,
+ ...(infoJson ? { infoJson } : {}),
+ ...(descriptionTxt ? { descriptionTxt } : {}),
+ ...(sourceTxt ? { sourceTxt } : {}),
+ };
+ const folderId = folder ? idFromFolderName(name) : null;
+ if (folderId) {
+ out.push({ folder, name, id: folderId, idFrom: "folder", media, ...base });
+ continue;
+ }
+ // No id on the folder (or the archive's root): every media file is its own
+ // video, by its file name. Sidecars are shared only when there is one.
+ if (media.length === 0) {
+ out.push({ folder, name, id: null, idFrom: null, media, ...base });
+ continue;
+ }
+ for (const m of media) {
+ const id = idFromMediaName(m.path);
+ out.push({
+ folder,
+ name,
+ id,
+ idFrom: id ? "file" : null,
+ media: [m],
+ ...(media.length === 1 ? base : { lost }),
+ });
+ }
+ }
+ return out;
+}
+
+// The archive's title for a folder named `NNN - <title> - (<id>)`, with any
+// `[LOST]` mark dropped.
+export function titleFromFolderName(name: string): string {
+ return name
+ .replace(FOLDER_ID_RE, "")
+ .replace(/\s*-\s*$/, "")
+ .replace(/^\s*\d+\s*(?:\[LOST\]\s*)?-\s*/i, "")
+ .replace(/\[LOST\]/gi, "")
+ .trim();
+}
+
+const MONTHS = [
+ "january", "february", "march", "april", "may", "june",
+ "july", "august", "september", "october", "november", "december",
+];
+
+// "Published on July 13th 2018" (the archive's description.txt) → "20180713".
+export function publishedDateFromText(text: string): string | null {
+ const m = /published\s+on\s+([A-Za-z]+)\s+(\d{1,2})(?:st|nd|rd|th)?,?\s+(\d{4})/i.exec(text);
+ if (!m) return null;
+ const month = MONTHS.indexOf(m[1].toLowerCase());
+ const day = Number(m[2]);
+ if (month < 0 || day < 1 || day > 31) return null;
+ return `${m[3]}${String(month + 1).padStart(2, "0")}${String(day).padStart(2, "0")}`;
+}
+
+// The description a record gets from description.txt: the text without the
+// "Published on …" line (that is the record's upload_date).
+export function descriptionFromText(text: string): string {
+ return text
+ .split(/\r?\n/)
+ .filter((l) => !/^\s*published\s+on\s+/i.test(l))
+ .join("\n")
+ .trim();
+}
+
+// A metadata.info.json for a video the channel does not hold, built from the
+// folder's description.txt and name when it has no yt-dlp .info.json. Title
+// first (a reader scans the head for it), upload_date last (another scans the
+// tail).
+export function archiveFolderInfoJson(opts: {
+ id: string;
+ folderName: string;
+ descriptionText?: string;
+ platform?: string;
+ durationSec?: number | null;
+}): Record<string, unknown> {
+ const youtube = opts.platform === undefined || opts.platform === "youtube";
+ const date = opts.descriptionText ? publishedDateFromText(opts.descriptionText) : null;
+ const description = opts.descriptionText ? descriptionFromText(opts.descriptionText) : "";
+ const url = youtube ? `https://www.youtube.com/watch?v=${opts.id}` : undefined;
+ return {
+ id: opts.id,
+ title: titleFromFolderName(opts.folderName) || opts.id,
+ ...(description ? { description } : {}),
+ ...(typeof opts.durationSec === "number" && Number.isFinite(opts.durationSec) && opts.durationSec > 0
+ ? { duration: Math.round(opts.durationSec * 100) / 100 }
+ : {}),
+ ...(url ? { webpage_url: url, original_url: url } : {}),
+ ...(youtube ? { extractor: "youtube", extractor_key: "Youtube" } : {}),
+ ...(date ? { upload_date: date } : {}),
+ };
+}
+
+// --- classification ---------------------------------------------------------
+
+export const ATTACH_ROW_CLASSES = [
+ // Held, media found, no pointer (or `replace`): attached.
+ "attach",
+ // Not held, media found, `createRecords`: a record is written, then attached.
+ "create",
+ // Held and already has a saved container: left alone without `replace`.
+ "already-attached",
+ // Media found for a video the channel does not hold (no `createRecords`).
+ "not-held",
+ // A `[LOST]` folder: listed, never attached.
+ "lost",
+ // A folder with an id but no video file.
+ "no-media",
+ // Media with no id the rules can read.
+ "unmatched",
+ // More than one video file for one id: name the file with `items`.
+ "ambiguous",
+] as const;
+export type AttachRowClass = (typeof ATTACH_ROW_CLASSES)[number];
+
+export type AttachRow = {
+ class: AttachRowClass;
+ id: string | null;
+ folder: string;
+ // The file to copy ("attach"/"create"), or the candidates ("ambiguous").
+ media?: SourceFile;
+ candidates?: SourceFile[];
+ // The folder, for createRecords.
+ source?: ArchiveFolder;
+ // `attach` over an existing pointer (replace: true).
+ replacing?: boolean;
+};
+
+export type AttachPlan = {
+ rows: AttachRow[];
+ // Held videos no folder supplied media for (only when the whole archive was
+ // considered: no `items`, no `match`).
+ heldWithoutMedia: string[] | null;
+};
+
+export type AttachItem = { id: string; path: string };
+
+export function planAttach(opts: {
+ folders: readonly ArchiveFolder[];
+ held: ReadonlySet<string>;
+ attached: ReadonlySet<string>;
+ createRecords?: boolean;
+ replace?: boolean;
+ match?: RegExp;
+ // Explicit id → file pairs. When given, they are the whole plan.
+ items?: readonly AttachItem[];
+ // Every file in the archive, to resolve `items` paths.
+ files?: readonly SourceFile[];
+}): AttachPlan {
+ const classify = (id: string, media: SourceFile, folder: string, source?: ArchiveFolder): AttachRow => {
+ if (opts.held.has(id)) {
+ if (opts.attached.has(id) && !opts.replace) {
+ return { class: "already-attached", id, folder, media };
+ }
+ return {
+ class: "attach",
+ id,
+ folder,
+ media,
+ ...(source ? { source } : {}),
+ ...(opts.attached.has(id) ? { replacing: true } : {}),
+ };
+ }
+ return {
+ class: opts.createRecords ? "create" : "not-held",
+ id,
+ folder,
+ media,
+ ...(source ? { source } : {}),
+ };
+ };
+
+ if (opts.items) {
+ const byPath = new Map((opts.files ?? []).map((f) => [f.path, f]));
+ const folders = new Map(opts.folders.map((f) => [f.folder, f]));
+ const rows = opts.items.map((item): AttachRow => {
+ const media = byPath.get(item.path);
+ const folder = dirName(item.path);
+ if (!media) return { class: "no-media", id: item.id, folder };
+ return classify(item.id, media, folder, folders.get(folder));
+ });
+ return { rows, heldWithoutMedia: null };
+ }
+
+ const considered = opts.match
+ ? opts.folders.filter(
+ (f) => opts.match!.test(f.folder) || f.media.some((m) => opts.match!.test(m.path)),
+ )
+ : opts.folders;
+
+ // Two folders claiming one id: neither is attached on a guess.
+ const idCount = new Map<string, number>();
+ for (const f of considered) {
+ if (f.id && f.media.length > 0 && !f.lost) idCount.set(f.id, (idCount.get(f.id) ?? 0) + 1);
+ }
+
+ const rows: AttachRow[] = [];
+ for (const f of considered) {
+ if (f.lost) {
+ rows.push({ class: "lost", id: f.id, folder: f.folder });
+ continue;
+ }
+ if (!f.id) {
+ if (f.media.length > 0) {
+ rows.push({ class: "unmatched", id: null, folder: f.folder, candidates: f.media });
+ }
+ continue;
+ }
+ if (f.media.length === 0) {
+ rows.push({ class: "no-media", id: f.id, folder: f.folder });
+ continue;
+ }
+ if (f.media.length > 1 || (idCount.get(f.id) ?? 0) > 1) {
+ rows.push({ class: "ambiguous", id: f.id, folder: f.folder, candidates: f.media });
+ continue;
+ }
+ rows.push(classify(f.id, f.media[0], f.folder, f));
+ }
+
+ let heldWithoutMedia: string[] | null = null;
+ if (!opts.match) {
+ const supplied = new Set(
+ rows.filter((r) => r.media && r.id).map((r) => r.id as string),
+ );
+ for (const r of rows) if (r.class === "ambiguous" && r.id) supplied.add(r.id);
+ heldWithoutMedia = [...opts.held].filter((id) => !supplied.has(id)).sort();
+ }
+ return { rows, heldWithoutMedia };
+}
+
+export function countAttachRows(rows: readonly AttachRow[]): Record<AttachRowClass, number> {
+ const out = Object.fromEntries(ATTACH_ROW_CLASSES.map((c) => [c, 0])) as Record<AttachRowClass, number>;
+ for (const r of rows) out[r.class] += 1;
+ return out;
+}
diff --git a/common/lib/lastResortSeeder.test.ts b/common/lib/lastResortSeeder.test.ts
@@ -0,0 +1,162 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import {
+ initialSeederMemory,
+ otherSeeders,
+ stepSeeder,
+ type SeederMemory,
+ type SeederPolicy,
+ type SeederTransition,
+ type SwarmObservation,
+} from "./lastResortSeeder";
+
+// Run with: pnpm --filter yt-dlp-transcript-common exec tsx --test lib/lastResortSeeder.test.ts
+//
+// The peer-of-last-resort state machine against a fake scrape source: a
+// scripted swarm, polled every POLL seconds, with the seeder's own
+// announce state fed back in (counted in `complete` while it seeds).
+
+const POLL = 30;
+const POLICY: SeederPolicy = { standbyAfterSeconds: 120, resumeAfterSeconds: 300 };
+
+type Tick = {
+ // Other seeders and leechers the tracker knows (the seeder itself is added
+ // by the harness while it seeds). null = the scrape failed.
+ others: number | null;
+ leechers?: number;
+ connectedSeeds?: number;
+ connectedLeechers?: number;
+};
+
+// Run a script; return each poll's state and every transition with its time.
+function run(script: Tick[], policy = POLICY, start: SeederMemory = initialSeederMemory(0)) {
+ let mem = start;
+ const states: string[] = [];
+ const transitions: (SeederTransition & { at: number })[] = [];
+ script.forEach((t, i) => {
+ const at = (i + 1) * POLL * 1000;
+ const self = mem.state === "seeding";
+ const o: SwarmObservation = {
+ at,
+ scrape: t.others === null ? null : { complete: t.others + (self ? 1 : 0), incomplete: t.leechers ?? 0 },
+ selfCounted: self,
+ connectedSeeds: t.connectedSeeds ?? 0,
+ connectedLeechers: t.connectedLeechers ?? 0,
+ };
+ const step = stepSeeder(mem, o, policy);
+ if (step.transition) transitions.push({ ...step.transition, at: at / 1000 });
+ mem = step.memory;
+ states.push(mem.state === "seeding" ? "S" : "-");
+ });
+ return { states: states.join(""), transitions, memory: mem };
+}
+
+const rep = (n: number, t: Tick) => Array.from({ length: n }, () => t);
+
+test("alone: it seeds and stays seeding", () => {
+ const r = run(rep(20, { others: 0, leechers: 1 }));
+ assert.equal(r.states, "S".repeat(20));
+ assert.deepEqual(r.transitions, []);
+});
+
+test("others present for standbyAfterSeconds → standby, not before", () => {
+ // 120 s at a 30 s poll: the run starts at the 1st poll (t=30) and is long
+ // enough at the 5th (t=150).
+ const r = run(rep(6, { others: 1 }));
+ assert.equal(r.states, "SSSS--");
+ assert.equal(r.transitions.length, 1);
+ assert.equal(r.transitions[0].to, "standby");
+ assert.equal(r.transitions[0].at, 150);
+ assert.match(r.transitions[0].reason, /1 other seeder\(s\) on every poll for 120s/);
+});
+
+test("hysteresis on the standby edge: one poll with no other seeder restarts the clock", () => {
+ const r = run([...rep(3, { others: 1 }), { others: 0 }, ...rep(4, { others: 1 }), ...rep(2, { others: 1 })]);
+ // The clock restarts at poll 5 (t=150) and is long enough at poll 9 (t=270).
+ assert.equal(r.transitions[0]?.at, 270);
+});
+
+test("its own announce is not another seeder", () => {
+ // The scrape counts it (complete = 1) while it seeds: that is no one else.
+ const r = run(rep(10, { others: 0 }));
+ assert.equal(r.states, "S".repeat(10));
+});
+
+test("standby → seeding at once when no other seeder remains and a leecher waits", () => {
+ const r = run([...rep(5, { others: 1 }), { others: 1 }, { others: 0, leechers: 1 }]);
+ assert.equal(r.transitions.map((t) => t.to).join(","), "standby,seeding");
+ assert.equal(r.transitions[1].at, 210);
+ assert.match(r.transitions[1].reason, /no other seeder and 1 leecher\(s\) waiting/);
+});
+
+test("a connected leecher with no other source counts as waiting", () => {
+ const start: SeederMemory = { state: "standby", since: 0, othersSince: null, aloneSince: null, silentSince: null };
+ const r = run([{ others: 0, leechers: 0, connectedLeechers: 1 }], POLICY, start);
+ assert.equal(r.transitions[0]?.to, "seeding");
+});
+
+test("hysteresis on the resume edge: alone with nobody waiting resumes after resumeAfterSeconds", () => {
+ const start: SeederMemory = { state: "standby", since: 0, othersSince: null, aloneSince: null, silentSince: null };
+ // 300 s at 30 s: the run starts at t=30 and is long enough at t=330 (poll 11).
+ const r = run(rep(12, { others: 0, leechers: 0 }), POLICY, start);
+ assert.equal(r.transitions.length, 1);
+ assert.equal(r.transitions[0].at, 330);
+ assert.match(r.transitions[0].reason, /no other seeder on every poll for 300s/);
+ // A poll that sees a seeder again restarts that clock.
+ const r2 = run([...rep(5, { others: 0 }), { others: 1 }, ...rep(5, { others: 0 })], POLICY, start);
+ assert.deepEqual(r2.transitions, []);
+});
+
+test("a flapping swarm does not flap the seeder", () => {
+ // Another seeder that comes and goes every other poll, with a leecher.
+ const flap = Array.from({ length: 40 }, (_, i) => ({ others: i % 2 === 0 ? 1 : 0, leechers: 1 }));
+ const r = run(flap);
+ // Never two polls in a row with others → the standby clock never fills.
+ assert.deepEqual(r.transitions, []);
+ assert.equal(r.states, "S".repeat(40));
+});
+
+test("after a resume the standby edge needs its whole window again", () => {
+ const r = run([...rep(5, { others: 1 }), { others: 0, leechers: 1 }, ...rep(5, { others: 1 })]);
+ // standby at t=150, seeding at t=180, standby again only at t=180+150=330.
+ assert.deepEqual(r.transitions.map((t) => [t.to, t.at]), [["standby", 150], ["seeding", 180], ["standby", 330]]);
+});
+
+test("a failed scrape decides nothing while seeding", () => {
+ const r = run([...rep(3, { others: 1 }), { others: null }, ...rep(5, { others: 1 })]);
+ // The silent poll breaks the others-run: standby needs 120 s from t=150.
+ assert.deepEqual(r.transitions.map((t) => t.at), [270]);
+});
+
+test("in standby, trackers silent for resumeAfterSeconds → resume (assume no other seeder)", () => {
+ const start: SeederMemory = { state: "standby", since: 0, othersSince: null, aloneSince: null, silentSince: null };
+ const r = run(rep(11, { others: null }), POLICY, start);
+ assert.equal(r.transitions[0]?.to, "seeding");
+ assert.equal(r.transitions[0]?.at, 330);
+ assert.match(r.transitions[0]?.reason ?? "", /no tracker answered/);
+});
+
+test("a connected seed is another source even when the scrape misses it", () => {
+ assert.equal(
+ otherSeeders({ at: 0, scrape: { complete: 1, incomplete: 0 }, selfCounted: true, connectedSeeds: 2, connectedLeechers: 0 }),
+ 2,
+ );
+ assert.equal(otherSeeders({ at: 0, scrape: null, selfCounted: false, connectedSeeds: 0, connectedLeechers: 3 }), null);
+ assert.equal(otherSeeders({ at: 0, scrape: null, selfCounted: false, connectedSeeds: 1, connectedLeechers: 0 }), 1);
+ assert.equal(otherSeeders({ at: 0, scrape: { complete: 0, incomplete: 0 }, selfCounted: true, connectedSeeds: 0, connectedLeechers: 0 }), 0);
+ const r = run(rep(6, { others: 0, connectedSeeds: 1 }));
+ assert.equal(r.transitions[0]?.to, "standby");
+});
+
+test("the rollout's check: a second browser seeds → standby; it closes → resume", () => {
+ // Step 7 of the plan's rollout, as a script: a viewer finishes and seeds,
+ // the home seeder stands by; the viewer leaves and another viewer arrives.
+ const r = run([
+ ...rep(2, { others: 0, leechers: 1, connectedLeechers: 1 }),
+ ...rep(6, { others: 1, leechers: 0, connectedSeeds: 1 }),
+ ...rep(2, { others: 1 }),
+ { others: 0, leechers: 1 },
+ ]);
+ assert.deepEqual(r.transitions.map((t) => t.to), ["standby", "seeding"]);
+ assert.equal(r.memory.state, "seeding");
+});
diff --git a/common/lib/lastResortSeeder.ts b/common/lib/lastResortSeeder.ts
@@ -0,0 +1,153 @@
+// THE PEER OF LAST RESORT (release 21 D4b): when the home seeder seeds a
+// torrent, and when it stands by.
+//
+// The home seeder exists so that a torrent with nobody else seeding it still
+// plays. It should not be the swarm's workhorse: every byte it uploads goes
+// through a VPN on a home connection. So per torrent it SEEDS only while no
+// other seeder is known, and STANDS BY (stops announcing, closes its peers,
+// keeps the data) while others are. This module is the decision and nothing
+// else — pure, clocked by the caller, so every edge is testable against a
+// fake scrape (lastResortSeeder.test.ts); `archilyzer seed`
+// (controller/seeder.ts) observes the swarm and applies what it returns.
+//
+// WHAT IT SEES, per torrent per poll (`SwarmObservation`):
+// - the trackers' scrape: `complete` (seeders) and `incomplete` (leechers),
+// or null when no tracker answered;
+// - whether the seeder itself is counted in that `complete` (it is while it
+// announces as a seeder; once it has sent `stopped`, it is not);
+// - its own wires: connected peers that have every piece (other sources)
+// and connected peers that do not (leechers).
+//
+// THE EDGES, both with hysteresis so a swarm hovering at one seeder does not
+// make it flap:
+// seeding → standby other seeders seen on EVERY poll for
+// `standbyAfterSeconds` (the clock restarts the moment
+// a poll sees none);
+// standby → seeding no other seeder AND a leecher waiting (the tracker's
+// `incomplete`, or a wire): at once — someone is trying
+// to play it now;
+// no other seeder on every poll for `resumeAfterSeconds`
+// with nobody waiting: so the torrent is never left with
+// no seeder for longer than that.
+// A transition restarts both clocks, so the reverse edge needs its full
+// window again. A failed scrape decides nothing (the state holds) — except
+// that a seeder in standby whose trackers have been silent for
+// `resumeAfterSeconds` resumes: it cannot know there is another seeder, and
+// being the last resort means assuming there is not.
+//
+// Known cost (the plan's): if the last other seeder leaves mid-stream, a
+// viewer stalls until the next poll sees it.
+
+export type SeederTorrentState = "seeding" | "standby";
+
+export type SwarmObservation = {
+ // Milliseconds (the caller's clock).
+ at: number;
+ scrape: { complete: number; incomplete: number } | null;
+ // Whether `scrape.complete` includes the seeder itself.
+ selfCounted: boolean;
+ // Connected peers holding every piece, and the rest.
+ connectedSeeds: number;
+ connectedLeechers: number;
+};
+
+export type SeederPolicy = {
+ standbyAfterSeconds: number;
+ resumeAfterSeconds: number;
+};
+
+export type SeederMemory = {
+ state: SeederTorrentState;
+ // When the current state began.
+ since: number;
+ // Start of the current run of polls that saw other seeders / saw none /
+ // got no scrape. Null when the last poll broke the run.
+ othersSince: number | null;
+ aloneSince: number | null;
+ silentSince: number | null;
+};
+
+export type SeederTransition = {
+ from: SeederTorrentState;
+ to: SeederTorrentState;
+ reason: string;
+};
+
+export type SeederStep = {
+ memory: SeederMemory;
+ transition: SeederTransition | null;
+ // What this poll saw, in words, for the log.
+ seen: string;
+};
+
+// A torrent starts SEEDING: on boot the seeder knows nothing about the swarm,
+// and the safe assumption for a last resort is that it is the only one.
+export function initialSeederMemory(at: number): SeederMemory {
+ return { state: "seeding", since: at, othersSince: null, aloneSince: null, silentSince: null };
+}
+
+// Other seeders this poll saw: the scrape's count without the seeder itself,
+// or the seeds connected to it, whichever is more (a seed on another tracker
+// the scrape did not ask is still a source). Null when there was no scrape
+// and no connected seed — unknown, not zero.
+export function otherSeeders(o: SwarmObservation): number | null {
+ const fromScrape = o.scrape ? Math.max(0, o.scrape.complete - (o.selfCounted ? 1 : 0)) : null;
+ if (fromScrape === null) return o.connectedSeeds > 0 ? o.connectedSeeds : null;
+ return Math.max(fromScrape, o.connectedSeeds);
+}
+
+function leechersWaiting(o: SwarmObservation): number {
+ return Math.max(o.scrape?.incomplete ?? 0, o.connectedLeechers);
+}
+
+const secs = (ms: number) => `${Math.round(ms / 1000)}s`;
+
+export function stepSeeder(
+ memory: SeederMemory,
+ o: SwarmObservation,
+ policy: SeederPolicy,
+): SeederStep {
+ const others = otherSeeders(o);
+ const waiting = leechersWaiting(o);
+ const seen =
+ others === null
+ ? `no scrape answered; ${o.connectedSeeds} seed(s) and ${o.connectedLeechers} leecher(s) connected`
+ : `${others} other seeder(s), ${waiting} leecher(s)`;
+
+ // Advance the three run clocks.
+ const m: SeederMemory = {
+ ...memory,
+ othersSince: others !== null && others > 0 ? (memory.othersSince ?? o.at) : null,
+ aloneSince: others === 0 ? (memory.aloneSince ?? o.at) : null,
+ silentSince: others === null ? (memory.silentSince ?? o.at) : null,
+ };
+ const standbyMs = policy.standbyAfterSeconds * 1000;
+ const resumeMs = policy.resumeAfterSeconds * 1000;
+ const go = (to: SeederTorrentState, reason: string): SeederStep => ({
+ memory: { state: to, since: o.at, othersSince: null, aloneSince: null, silentSince: null },
+ transition: { from: m.state, to, reason },
+ seen,
+ });
+
+ if (m.state === "seeding") {
+ if (m.othersSince !== null && o.at - m.othersSince >= standbyMs) {
+ return go(
+ "standby",
+ `${others} other seeder(s) on every poll for ${secs(o.at - m.othersSince)} (standby after ${policy.standbyAfterSeconds}s)`,
+ );
+ }
+ return { memory: m, transition: null, seen };
+ }
+
+ // standby
+ if (others === 0 && waiting > 0) {
+ return go("seeding", `no other seeder and ${waiting} leecher(s) waiting`);
+ }
+ if (m.aloneSince !== null && o.at - m.aloneSince >= resumeMs) {
+ return go("seeding", `no other seeder on every poll for ${secs(o.at - m.aloneSince)} (resume after ${policy.resumeAfterSeconds}s)`);
+ }
+ if (m.silentSince !== null && o.at - m.silentSince >= resumeMs) {
+ return go("seeding", `no tracker answered for ${secs(o.at - m.silentSince)}: assuming no other seeder`);
+ }
+ return { memory: m, transition: null, seen };
+}
diff --git a/common/lib/metadataHistory.ts b/common/lib/metadataHistory.ts
@@ -78,6 +78,10 @@ export const METADATA_HISTORY_WRITERS = [
// changes after the first fetch — a livestream VOD that gains its processed
// formats and auto-captions hours after it ends.
"refresh",
+ // A record created from a local archive's folder (release 21 D1,
+ // controller/attachMedia.ts `createRecords`): the folder's own yt-dlp
+ // .info.json, or one built from its description.txt / source.txt and name.
+ "local-archive",
] as const;
export type MetadataHistoryWriter = (typeof METADATA_HISTORY_WRITERS)[number];
diff --git a/common/lib/playableMedia.test.ts b/common/lib/playableMedia.test.ts
@@ -0,0 +1,140 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import {
+ MAX_PIECE_LENGTH,
+ MIN_PIECE_LENGTH,
+ decidePlayable,
+ magnetFor,
+ parsePlayableManifest,
+ parseProbe,
+ pieceLengthFor,
+ type MediaProbe,
+} from "./playableMedia";
+
+// Run with: pnpm --filter yt-dlp-transcript-common exec tsx --test lib/playableMedia.test.ts
+
+const p = (ext: string, vcodec: string | null, acodec: string | null): MediaProbe => ({
+ formatName: ext === "mp4" ? "mov,mp4,m4a,3gp,3g2,mj2" : "matroska,webm",
+ ext,
+ vcodec,
+ acodec,
+});
+
+// The whole table: video × audio × source container → the playable container.
+// The pilot archive's measured mix is every row marked ★.
+const TABLE: [string, string, string | null, "mp4" | "webm" | null][] = [
+ ["mp4", "h264", "aac", "mp4"], // ★ 145
+ ["mp4", "h264", "opus", "mp4"],
+ ["mp4", "vp9", "aac", "mp4"],
+ ["mp4", "vp9", "opus", "mp4"],
+ ["mp4", "av1", "aac", "mp4"], // ★ 1
+ ["mp4", "av1", "opus", "mp4"],
+ ["mkv", "h264", "aac", "mp4"],
+ ["mkv", "h264", "opus", "mp4"], // ★ 2
+ ["mkv", "vp9", "aac", "mp4"], // ★ 11
+ ["mkv", "vp9", "opus", "mp4"],
+ ["mkv", "av1", "aac", "mp4"],
+ ["mkv", "av1", "opus", "mp4"], // ★ 1
+ ["webm", "vp9", "opus", "webm"], // ★ 9
+ ["webm", "av1", "opus", "webm"],
+ ["webm", "vp8", "vorbis", "webm"],
+ ["webm", "vp9", null, "webm"],
+ ["webm", "h264", "aac", "mp4"], // not a webm pair: mp4
+ ["webm", "vp9", "aac", "mp4"],
+ ["mp4", "h264", null, "mp4"],
+ ["mp4", "h264", "mp3", "mp4"],
+ ["mkv", "hevc", "aac", null], // no lossless remux a browser plays
+ ["mkv", "mpeg4", "mp3", null],
+ ["mkv", "h264", "ac3", null],
+ ["webm", "vp8", "aac", null],
+];
+
+test("the remux decision table", () => {
+ for (const [ext, v, a, want] of TABLE) {
+ const d = decidePlayable(p(ext, v, a));
+ const label = `${v}/${a ?? "none"} in ${ext}`;
+ if (want === null) {
+ assert.equal(d.playable, false, label);
+ continue;
+ }
+ assert.equal(d.playable, true, label);
+ if (!d.playable) continue;
+ assert.equal(d.ext, want, label);
+ assert.equal(d.mime, `video/${want}`, label);
+ // Lossless always; the index in front always.
+ assert.ok(d.args.includes("copy"), label);
+ assert.ok(!d.args.some((x) => /^lib|^-crf$|^-b:/.test(x)), `${label}: no encoder`);
+ if (want === "mp4") assert.ok(d.args.join(" ").includes("-movflags +faststart"), label);
+ if (want === "webm") assert.ok(d.args.join(" ").includes("-cues_to_front 1"), label);
+ }
+});
+
+test("no video is not playable; metadata and extra streams are dropped", () => {
+ assert.equal(decidePlayable(p("mp4", null, "aac")).playable, false);
+ const d = decidePlayable(p("mp4", "h264", "aac"));
+ assert.ok(d.playable);
+ if (!d.playable) return;
+ const a = d.args.join(" ");
+ assert.match(a, /-map 0:V:0 -map 0:a:0\?/, "first real video (not cover art), first audio if any");
+ assert.match(a, /-map_metadata -1/);
+ assert.match(a, /-map_chapters -1/);
+ assert.match(a, /-sn -dn/);
+ assert.match(a, /-fflags \+bitexact/);
+});
+
+test("parseProbe skips cover art and reads the extension", () => {
+ const probe = parseProbe(
+ {
+ format: { format_name: "matroska,webm" },
+ streams: [
+ { codec_type: "video", codec_name: "mjpeg", disposition: { attached_pic: 1 } },
+ { codec_type: "video", codec_name: "vp9", disposition: { attached_pic: 0 } },
+ { codec_type: "audio", codec_name: "opus" },
+ { codec_type: "subtitle", codec_name: "ass" },
+ ],
+ },
+ ".WEBM",
+ );
+ assert.deepEqual(probe, { formatName: "matroska,webm", ext: "webm", vcodec: "vp9", acodec: "opus" });
+ assert.deepEqual(parseProbe(null, "mp4"), { formatName: "", ext: "mp4", vcodec: null, acodec: null });
+});
+
+test("piece length: a power of two, 256 KiB–1 MiB, scaled to the size", () => {
+ assert.equal(pieceLengthFor(1), MIN_PIECE_LENGTH);
+ assert.equal(pieceLengthFor(20 * 1024 ** 2), MIN_PIECE_LENGTH);
+ assert.equal(pieceLengthFor(800 * 1024 ** 2), 512 * 1024);
+ assert.equal(pieceLengthFor(2 * 1024 ** 3), MAX_PIECE_LENGTH);
+ assert.equal(pieceLengthFor(50 * 1024 ** 3), MAX_PIECE_LENGTH);
+ for (const n of [1e6, 3e8, 7e8, 1.5e9]) {
+ const len = pieceLengthFor(n);
+ assert.equal(len & (len - 1), 0, `${len} is a power of two`);
+ }
+});
+
+test("a magnet names the infohash, the file and only the trackers given", () => {
+ const hash = "a".repeat(40);
+ assert.equal(magnetFor({ infoHash: hash, name: "abc.mp4" }), `magnet:?xt=urn:btih:${hash}&dn=abc.mp4`);
+ assert.equal(
+ magnetFor({ infoHash: hash, name: "a b.mp4", trackers: ["wss://t.example/announce"] }),
+ `magnet:?xt=urn:btih:${hash}&dn=a%20b.mp4&tr=wss%3A%2F%2Ft.example%2Fannounce`,
+ );
+});
+
+test("the manifest parse is tolerant: a bad entry is dropped, not fatal", () => {
+ const good = {
+ sourceSha256: "b".repeat(64),
+ file: "x/x.mp4",
+ torrent: "x/x.torrent",
+ ext: "mp4",
+ mime: "video/mp4",
+ bytes: 10,
+ infoHash: "c".repeat(40),
+ pieceLength: 262144,
+ vcodec: "h264",
+ acodec: "aac",
+ preparedAt: "2026-10-09T00:00:00Z",
+ };
+ const m = parsePlayableManifest({ version: 1, slug: "s", entries: { x: good, y: { ...good, infoHash: "nope" } } }, "s");
+ assert.deepEqual(Object.keys(m.entries), ["x"]);
+ assert.deepEqual(parsePlayableManifest("garbage", "s").entries, {});
+});
diff --git a/common/lib/playableMedia.ts b/common/lib/playableMedia.ts
@@ -0,0 +1,218 @@
+// A SAVED CONTAINER, MADE PLAYABLE IN A BROWSER — WITHOUT RE-ENCODING
+// (release 21 D3).
+//
+// The sites play an archived video in a native <video> fed by WebTorrent
+// (D5). A browser plays mp4 (h264/vp9/av1 with aac/opus/mp3) and webm
+// (vp9/vp8/av1 with opus/vorbis); it does not play Matroska, and an mp4 whose
+// `moov` atom is at the back cannot start until the file's last pieces
+// arrive. So each container is REMUXED — `ffmpeg -c copy`, every frame and
+// sample bit-for-bit — into the browser's container with its index in front:
+//
+// video audio source → playable
+// h264/vp9/av1 aac/opus/mp3/none mp4/mkv/… → mp4 (+faststart)
+// vp9/vp8/av1 opus/vorbis/none webm → webm (cues in front)
+// anything else (hevc, mpeg4, …, or no video) → not playable: a
+// re-encode is a ruling,
+// not a default
+//
+// The pilot archive's measured mix (release 21): 145 mp4 h264/aac (124 of
+// them moov-at-back), 9 webm vp9/opus, 11 mkv vp9/aac, 2 mkv h264/opus, 1 mp4
+// av1/aac, 1 mkv av1/opus — every one of them lossless under this table.
+//
+// Only the first real video stream (not cover art) and the first audio stream
+// are kept; subtitles, data, attachments, global metadata and chapters are
+// dropped — a published file carries the picture and the sound and nothing
+// that names anyone. `+bitexact` keeps ffmpeg's version string out, so the
+// same source remuxes to the same bytes, and so to the same infohash.
+//
+// PURE: the probe is parsed here from ffprobe's JSON; running ffprobe and
+// ffmpeg is the controller's (controller/preparePlayable.ts).
+
+export type ProbedStream = {
+ codec_type?: string;
+ codec_name?: string;
+ disposition?: { attached_pic?: number };
+};
+
+export type MediaProbe = {
+ // ffprobe's format_name ("mov,mp4,m4a,3gp,3g2,mj2", "matroska,webm").
+ formatName: string;
+ // The source file's extension, lower-case, no dot: ffprobe says
+ // "matroska,webm" for both an .mkv and a .webm, and only the name tells
+ // them apart.
+ ext: string;
+ vcodec: string | null;
+ acodec: string | null;
+};
+
+export type PlayableDecision =
+ | {
+ playable: true;
+ ext: "mp4" | "webm";
+ mime: "video/mp4" | "video/webm";
+ // The ffmpeg arguments between `-i <src>` and the output path.
+ args: string[];
+ reason: string;
+ }
+ | { playable: false; reason: string };
+
+const MP4_VIDEO = new Set(["h264", "vp9", "av1"]);
+const MP4_AUDIO = new Set(["aac", "opus", "mp3"]);
+const WEBM_VIDEO = new Set(["vp9", "vp8", "av1"]);
+const WEBM_AUDIO = new Set(["opus", "vorbis"]);
+
+// ffprobe -show_streams -show_format -of json → the probe.
+export function parseProbe(json: unknown, ext: string): MediaProbe {
+ const j = (json ?? {}) as { streams?: ProbedStream[]; format?: { format_name?: string } };
+ const streams = Array.isArray(j.streams) ? j.streams : [];
+ const video = streams.find((s) => s.codec_type === "video" && !s.disposition?.attached_pic);
+ const audio = streams.find((s) => s.codec_type === "audio");
+ return {
+ formatName: j.format?.format_name ?? "",
+ ext: ext.replace(/^\./, "").toLowerCase(),
+ vcodec: video?.codec_name ?? null,
+ acodec: audio?.codec_name ?? null,
+ };
+}
+
+const COMMON_ARGS = [
+ "-map", "0:V:0",
+ "-map", "0:a:0?",
+ "-c", "copy",
+ "-sn", "-dn",
+ "-map_metadata", "-1",
+ "-map_chapters", "-1",
+ "-fflags", "+bitexact",
+ "-flags:v", "+bitexact",
+ "-flags:a", "+bitexact",
+];
+
+export function decidePlayable(p: MediaProbe): PlayableDecision {
+ const v = p.vcodec;
+ const a = p.acodec;
+ const what = `${v ?? "no video"}/${a ?? "no audio"} in ${p.ext || p.formatName}`;
+ if (!v) return { playable: false, reason: `${what}: no video stream` };
+ const isWebm = p.ext === "webm";
+ if (isWebm && WEBM_VIDEO.has(v) && (a === null || WEBM_AUDIO.has(a))) {
+ return {
+ playable: true,
+ ext: "webm",
+ mime: "video/webm",
+ args: [...COMMON_ARGS, "-f", "webm", "-cues_to_front", "1"],
+ reason: `${what} → webm, remuxed with its cues in front`,
+ };
+ }
+ if (MP4_VIDEO.has(v) && (a === null || MP4_AUDIO.has(a))) {
+ return {
+ playable: true,
+ ext: "mp4",
+ mime: "video/mp4",
+ args: [...COMMON_ARGS, "-f", "mp4", "-movflags", "+faststart"],
+ reason: `${what} → mp4, remuxed with its index in front`,
+ };
+ }
+ return {
+ playable: false,
+ reason: `${what}: no lossless remux a browser plays (a re-encode is an operator ruling)`,
+ };
+}
+
+// --- torrents ---------------------------------------------------------------
+
+export const MIN_PIECE_LENGTH = 256 * 1024;
+export const MAX_PIECE_LENGTH = 1024 * 1024;
+// Aim for about this many pieces: a few thousand is what a browser peer
+// handles comfortably, and a seek fetches at most one piece of slack.
+const TARGET_PIECES = 2048;
+
+// A power of two between 256 KiB and 1 MiB, scaled to the file: a 20 MB clip
+// gets 256 KiB pieces, a 2 GB episode 1 MiB.
+export function pieceLengthFor(bytes: number): number {
+ let len = MIN_PIECE_LENGTH;
+ while (len < MAX_PIECE_LENGTH && bytes / len > TARGET_PIECES) len *= 2;
+ return len;
+}
+
+// A magnet for an infohash, its display name and the announce URLs a site
+// names. The trackers are not in the info dict, so the same infohash can be
+// announced anywhere later.
+export function magnetFor(opts: { infoHash: string; name: string; trackers?: readonly string[] }): string {
+ const parts = [`xt=urn:btih:${opts.infoHash}`, `dn=${encodeURIComponent(opts.name)}`];
+ for (const tr of opts.trackers ?? []) parts.push(`tr=${encodeURIComponent(tr)}`);
+ return `magnet:?${parts.join("&")}`;
+}
+
+// --- the manifest -----------------------------------------------------------
+
+// `<root>/<slug>/playable.json`: what has been prepared, keyed by video id.
+// A re-run compares `sourceSha256` and skips an entry whose files are there.
+export type PlayableEntry = {
+ // The saved container this was made from, by hash.
+ sourceSha256: string;
+ // Relative to `<root>/<slug>/`.
+ file: string;
+ torrent: string;
+ ext: "mp4" | "webm";
+ mime: "video/mp4" | "video/webm";
+ bytes: number;
+ infoHash: string;
+ pieceLength: number;
+ vcodec: string;
+ acodec: string | null;
+ preparedAt: string;
+};
+
+export type PlayableManifest = {
+ version: 1;
+ slug: string;
+ entries: Record<string, PlayableEntry>;
+};
+
+export const PLAYABLE_MANIFEST_FILENAME = "playable.json";
+
+export function emptyPlayableManifest(slug: string): PlayableManifest {
+ return { version: 1, slug, entries: {} };
+}
+
+const HEX40 = /^[0-9a-f]{40}$/;
+const HEX64 = /^[0-9a-f]{64}$/;
+
+// Tolerant: a malformed entry is dropped (and so re-prepared), never fatal.
+export function parsePlayableManifest(raw: unknown, slug: string): PlayableManifest {
+ const out = emptyPlayableManifest(slug);
+ if (!raw || typeof raw !== "object" || Array.isArray(raw)) return out;
+ const entries = (raw as { entries?: unknown }).entries;
+ if (!entries || typeof entries !== "object" || Array.isArray(entries)) return out;
+ for (const [id, e] of Object.entries(entries as Record<string, unknown>)) {
+ if (!e || typeof e !== "object") continue;
+ const r = e as Record<string, unknown>;
+ if (
+ typeof r.sourceSha256 !== "string" || !HEX64.test(r.sourceSha256) ||
+ typeof r.file !== "string" || typeof r.torrent !== "string" ||
+ (r.ext !== "mp4" && r.ext !== "webm") ||
+ (r.mime !== "video/mp4" && r.mime !== "video/webm") ||
+ typeof r.bytes !== "number" ||
+ typeof r.infoHash !== "string" || !HEX40.test(r.infoHash) ||
+ typeof r.pieceLength !== "number" ||
+ typeof r.vcodec !== "string" ||
+ (r.acodec !== null && typeof r.acodec !== "string") ||
+ typeof r.preparedAt !== "string"
+ ) {
+ continue;
+ }
+ out.entries[id] = {
+ sourceSha256: r.sourceSha256,
+ file: r.file,
+ torrent: r.torrent,
+ ext: r.ext,
+ mime: r.mime,
+ bytes: r.bytes,
+ infoHash: r.infoHash,
+ pieceLength: r.pieceLength,
+ vcodec: r.vcodec,
+ acodec: r.acodec as string | null,
+ preparedAt: r.preparedAt,
+ };
+ }
+ return out;
+}
diff --git a/common/lib/savedVideo-server.ts b/common/lib/savedVideo-server.ts
@@ -118,11 +118,21 @@ export async function persistSourceVideo(opts: {
// `saved-videos` as a real directory in the instant between the mover's
// rename and its symlink.
paths?: Pick<Paths, "transcriptsDir" | "savedVideosDir">;
+ // THE FILE TO MOVE, when it is not `<videoDir>/<sourceFilename>`. A container
+ // attached from a local archive (controller/attachMedia.ts) is copied out of
+ // the archive into a staging file BESIDE `storeDir`, on the store's disk, so
+ // the move is a same-filesystem rename — copying 36 GB through the corpus
+ // SSD first would be the wrong disk twice. The stored name is still
+ // `sourceFilename`.
+ sourcePath?: string;
+ // The bytes' sha256, when the caller already hashed them (an attach hashes
+ // while it copies); the backup step otherwise fills it in later.
+ sha256?: string;
}): Promise<SavedVideoPointer> {
if (opts.paths) {
await assertSavedVideosStoreWritable(opts.paths, opts.storeDir);
}
- const src = path.join(opts.videoDir, opts.sourceFilename);
+ const src = opts.sourcePath ?? path.join(opts.videoDir, opts.sourceFilename);
const dest = path.join(opts.storeDir, opts.sourceFilename);
const st = await stat(src);
await moveFileCrossDevice(src, dest);
@@ -132,6 +142,7 @@ export async function persistSourceVideo(opts: {
file: opts.sourceFilename,
bytes: st.size,
...(opts.keepReason ? { keepReason: opts.keepReason } : {}),
+ ...(opts.sha256 ? { sha256: opts.sha256 } : {}),
...(opts.origin ? { origin: opts.origin } : {}),
...(opts.format ? { format: opts.format } : {}),
};
diff --git a/common/lib/savedVideo.ts b/common/lib/savedVideo.ts
@@ -102,14 +102,32 @@ export function savedVideoFormatFromLine(
// The requester of a manually-sourced container: the tool, and the report clip
// it was wanted for. Deliberately the same vocabulary as ClipProvenance
// (lib/clipWindow.ts) so the video page renders both the same way.
+//
+// A CONTAINER ATTACHED FROM A LOCAL ARCHIVE (release 21 D1, `attach-media`)
+// carries `kind: "local-archive"` and says which archive, which entry in it,
+// the bytes' sha256 and when — the provenance of a file nobody downloaded.
+// `requestedBy` is still set (the tool, "attach-media"), so every reader that
+// renders "requested by" keeps working on it unchanged; a pointer without
+// `kind` is the requested-container shape it always was.
export type SavedVideoOrigin = {
requestedBy: string;
manifest?: string;
clipId?: string;
reason?: string;
requestedAt?: string;
+ kind?: SavedVideoOriginKind;
+ // local-archive: the archive (a .zip or .7z file, or a directory) as an
+ // absolute path, the entry's path inside it, the copied bytes' sha256 (hex)
+ // and when the attach wrote the pointer.
+ archive?: string;
+ entry?: string;
+ sha256?: string;
+ attachedAt?: string;
};
+export const SAVED_VIDEO_ORIGIN_KINDS = ["local-archive"] as const;
+export type SavedVideoOriginKind = (typeof SAVED_VIDEO_ORIGIN_KINDS)[number];
+
function parseOrigin(raw: unknown): SavedVideoOrigin | null {
if (!raw || typeof raw !== "object" || Array.isArray(raw)) return null;
const r = raw as Record<string, unknown>;
@@ -119,6 +137,18 @@ function parseOrigin(raw: unknown): SavedVideoOrigin | null {
const v = r[k];
if (typeof v === "string" && v !== "") out[k] = v;
}
+ // An unknown kind is dropped with its fields rather than failing the
+ // pointer: the container is still there, only its provenance goes unread.
+ if (
+ typeof r.kind === "string" &&
+ (SAVED_VIDEO_ORIGIN_KINDS as readonly string[]).includes(r.kind)
+ ) {
+ out.kind = r.kind as SavedVideoOriginKind;
+ for (const k of ["archive", "entry", "sha256", "attachedAt"] as const) {
+ const v = r[k];
+ if (typeof v === "string" && v !== "") out[k] = v;
+ }
+ }
return out;
}
diff --git a/common/lib/seederCompose.test.ts b/common/lib/seederCompose.test.ts
@@ -0,0 +1,80 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import { readFileSync } from "node:fs";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+
+// Run with: pnpm --filter yt-dlp-transcript-common exec tsx --test lib/seederCompose.test.ts
+//
+// THE KILL SWITCH IS THE CONSTRUCTION (release 21 D4b), so the construction is
+// what this pins, in docker-compose.seeder.yml: the seeder and the tracker
+// have no network of their own (they live in the VPN container's namespace),
+// publish nothing, and the VPN container's firewall is on and its WireGuard
+// config is the operator's — required, never defaulted, never baked.
+
+const REPO = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..", "..");
+const TEXT = readFileSync(path.join(REPO, "docker-compose.seeder.yml"), "utf8");
+
+// The services block as name → its lines (comments dropped). Two-space
+// service keys under a top-level `services:`; enough YAML for this file.
+function services(text: string): Map<string, string[]> {
+ const out = new Map<string, string[]>();
+ let inServices = false;
+ let current: string | null = null;
+ for (const raw of text.split("\n")) {
+ const line = raw.replace(/\s+#.*$/, "");
+ if (/^\S/.test(line)) {
+ inServices = /^services:\s*$/.test(line);
+ current = null;
+ continue;
+ }
+ if (!inServices || !line.trim() || /^\s*#/.test(raw)) continue;
+ const key = /^ {2}([a-z0-9-]+):\s*$/.exec(line);
+ if (key) {
+ current = key[1];
+ out.set(current, []);
+ } else if (current) {
+ out.get(current)!.push(line);
+ }
+ }
+ return out;
+}
+
+const S = services(TEXT);
+const block = (name: string) => (S.get(name) ?? []).join("\n");
+
+test("the profile has a VPN container, the seeder and the tracker — all behind profile `seeder`", () => {
+ assert.deepEqual([...S.keys()].sort(), ["seeder", "seeder-vpn", "tracker"]);
+ for (const name of S.keys()) assert.match(block(name), /^\s{4}profiles: \[seeder\]$/m, name);
+});
+
+test("the seeder and the tracker share the VPN container's network namespace and nothing else", () => {
+ for (const name of ["seeder", "tracker"]) {
+ const b = block(name);
+ assert.match(b, /^\s{4}network_mode: service:seeder-vpn$/m, `${name}: network_mode`);
+ assert.doesNotMatch(b, /^\s{4}networks:/m, `${name}: no networks of its own`);
+ assert.doesNotMatch(b, /^\s{4}ports:/m, `${name}: publishes nothing`);
+ assert.doesNotMatch(b, /^\s{4}network_mode: (host|bridge)/m, name);
+ // It waits for the tunnel to be up, not merely started.
+ assert.match(b, /seeder-vpn:\n\s+condition: service_healthy/, `${name}: waits for a healthy tunnel`);
+ // Read-only corpus: the seeder writes nothing.
+ for (const vol of b.matchAll(/^\s{6}- [^\s:]+:\/data\/[^\s:]+$/gm)) assert.fail(`${name}: writable mount ${vol[0]}`);
+ assert.match(b, /- corpus:\/data\/transcripts:ro$/m, `${name}: the corpus, read-only`);
+ }
+});
+
+test("the VPN container: WireGuard, the firewall on, the operator's config required and read-only, no port published", () => {
+ const b = block("seeder-vpn");
+ assert.match(b, /VPN_TYPE: wireguard/);
+ assert.match(b, /VPN_SERVICE_PROVIDER: custom/);
+ assert.match(b, /image: qmcgaw\/gluetun:v\d+\.\d+\.\d+$/m, "a pinned release");
+ assert.doesNotMatch(TEXT, /FIREWALL: "?off/i, "nothing turns a firewall off");
+ assert.match(b, /WIREGUARD_CONF_SECRETFILE: \/gluetun\/wireguard\/wg0\.conf/);
+ assert.doesNotMatch(b, /^\s{4}ports:/m);
+ assert.doesNotMatch(b, /network_mode/);
+ // `${VAR:?…}`: compose refuses to start without it; there is no default.
+ assert.match(b, /- \$\{SEEDER_WG_CONF:\?[^}]+\}:\/gluetun\/wireguard\/wg0\.conf:ro/);
+ assert.doesNotMatch(TEXT, /PrivateKey\s*=|WIREGUARD_PRIVATE_KEY:\s*\S/, "no key in the file");
+ // Inbound only when the operator forwards a port; empty by default.
+ assert.match(b, /FIREWALL_VPN_INPUT_PORTS: \$\{SEEDER_VPN_INPUT_PORTS:-\}/);
+});
diff --git a/common/lib/seederSettings.test.ts b/common/lib/seederSettings.test.ts
@@ -0,0 +1,40 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import { DEFAULT_SEEDER_SETTINGS, sanitizeSeeder, seederConfigured } from "./seederSettings";
+
+// Run with: pnpm --filter yt-dlp-transcript-common exec tsx --test lib/seederSettings.test.ts
+
+test("absent or garbage is the default, which is not configured", () => {
+ for (const v of [undefined, null, "x", 3, [], { sites: "jeralyzer-private" }]) {
+ const s = sanitizeSeeder(v);
+ assert.deepEqual(s.sites, []);
+ assert.equal(seederConfigured(s), false, JSON.stringify(v));
+ }
+ assert.deepEqual(sanitizeSeeder({}), DEFAULT_SEEDER_SETTINGS);
+});
+
+test("sites and trackers keep only well-formed, distinct entries", () => {
+ const s = sanitizeSeeder({
+ sites: ["jeralyzer-private", " jeralyzer-private ", "BAD ID", 3, "../x"],
+ trackers: ["wss://tracker.example/announce", "http://10.0.0.2:8000/announce", "udp://t.example:6969", "file:///etc/passwd", "not a url"],
+ });
+ assert.deepEqual(s.sites, ["jeralyzer-private"]);
+ assert.deepEqual(s.trackers, ["wss://tracker.example/announce", "http://10.0.0.2:8000/announce", "udp://t.example:6969"]);
+ assert.equal(seederConfigured(s), true);
+});
+
+test("numbers are clamped, and a bad interface name reads as unset", () => {
+ const s = sanitizeSeeder({
+ maxUploadKiBps: -5,
+ maxConnections: 10_000,
+ pollSeconds: 1,
+ standbyAfterSeconds: 10,
+ bindInterface: "wg0; rm -rf /",
+ });
+ assert.equal(s.maxUploadKiBps, 0);
+ assert.equal(s.maxConnections, 500);
+ assert.equal(s.pollSeconds, 10);
+ assert.equal(s.standbyAfterSeconds, 60);
+ assert.equal(s.bindInterface, "");
+ assert.equal(sanitizeSeeder({ bindInterface: " wg0 " }).bindInterface, "wg0");
+});
diff --git a/common/lib/seederSettings.ts b/common/lib/seederSettings.ts
@@ -0,0 +1,96 @@
+// settings.seeder — the home seeder of last resort (release 21 D4b).
+//
+// Pure: the type, its defaults, the sanitizer settingsSchema.ts runs it
+// through, and the per-field docs SETTINGS.md renders. The seeder itself is
+// controller/seeder.ts (`archilyzer seed`); the decision of when to seed is
+// lib/lastResortSeeder.ts.
+//
+// NOT CONFIGURED is `sites: []`, the default: `archilyzer seed` refuses to
+// start and `archilyzer doctor` says "seeder not configured".
+
+import type { FieldDocs } from "./fieldDocs";
+
+export type SeederSettings = {
+ sites: string[];
+ trackers: string[];
+ maxUploadKiBps: number;
+ maxConnections: number;
+ pollSeconds: number;
+ standbyAfterSeconds: number;
+ bindInterface: string;
+};
+
+export const DEFAULT_SEEDER_SETTINGS: SeederSettings = {
+ sites: [],
+ trackers: [],
+ maxUploadKiBps: 0,
+ maxConnections: 50,
+ pollSeconds: 60,
+ standbyAfterSeconds: 600,
+ bindInterface: "",
+};
+
+export const SEEDER_SETTINGS_FIELD_DOCS: FieldDocs<SeederSettings> = {
+ sites:
+ "The site ids whose playable torrents the seeder seeds: every channel of each site " +
+ "that has a playable manifest (`archilyzer media playable`). Empty = the seeder is " +
+ "not configured, and `archilyzer seed` refuses to start.",
+ trackers:
+ "Announce URLs the seeder announces to and scrapes (wss:// or ws:// for browsers, " +
+ "http(s):// or udp:// for desktop clients). Empty = each torrent's own announce list.",
+ maxUploadKiBps: "Upload cap across every torrent, KiB/s. 0 = unlimited.",
+ maxConnections: "Most peer connections per torrent. Clamped to [1, 500]; default 50.",
+ pollSeconds:
+ "How often the seeder scrapes the trackers and decides, per torrent, whether to " +
+ "seed or stand by. A viewer whose last other seeder leaves mid-stream waits up to " +
+ "this long. Clamped to [10, 3600]; default 60.",
+ standbyAfterSeconds:
+ "Other seeders seen on every poll for this long → the seeder stands by on that " +
+ "torrent (stops announcing, closes its peers, keeps the data). The same window, " +
+ "with no other seeder seen, brings it back when nobody is waiting; a waiting " +
+ "leecher brings it back at once. Clamped to [60, 86400]; default 600.",
+ bindInterface:
+ "The network interface the seeder must have — the VPN tunnel's (e.g. `wg0`, " +
+ "`tun0`). While it is missing the seeder does not start, and stops every torrent " +
+ "if it disappears. Empty = not checked (the docker profile `seeder` confines the " +
+ "process to the tunnel either way; see docker-compose.seeder.yml).",
+};
+
+function clampInt(value: unknown, fallback: number, min: number, max: number): number {
+ const n = typeof value === "number" && Number.isFinite(value) ? Math.floor(value) : fallback;
+ return Math.min(max, Math.max(min, n));
+}
+
+const SITE_ID_RE = /^[a-z0-9][a-z0-9-]*$/;
+const TRACKER_RE = /^(wss?|https?|udp):\/\/\S+$/i;
+const IFACE_RE = /^[A-Za-z0-9_.:-]{1,32}$/;
+
+function stringList(value: unknown, keep: (s: string) => boolean): string[] {
+ if (!Array.isArray(value)) return [];
+ const out: string[] = [];
+ for (const v of value) {
+ if (typeof v !== "string") continue;
+ const s = v.trim();
+ if (s && keep(s) && !out.includes(s)) out.push(s);
+ }
+ return out;
+}
+
+export function sanitizeSeeder(value: unknown): SeederSettings {
+ const d = DEFAULT_SEEDER_SETTINGS;
+ const r = (value && typeof value === "object" && !Array.isArray(value) ? value : {}) as Record<string, unknown>;
+ const iface = typeof r.bindInterface === "string" ? r.bindInterface.trim() : "";
+ return {
+ sites: stringList(r.sites, (s) => SITE_ID_RE.test(s)),
+ trackers: stringList(r.trackers, (s) => TRACKER_RE.test(s)),
+ maxUploadKiBps: clampInt(r.maxUploadKiBps, d.maxUploadKiBps, 0, 10_000_000),
+ maxConnections: clampInt(r.maxConnections, d.maxConnections, 1, 500),
+ pollSeconds: clampInt(r.pollSeconds, d.pollSeconds, 10, 3600),
+ standbyAfterSeconds: clampInt(r.standbyAfterSeconds, d.standbyAfterSeconds, 60, 86_400),
+ bindInterface: IFACE_RE.test(iface) ? iface : "",
+ };
+}
+
+export function seederConfigured(s: SeederSettings): boolean {
+ return s.sites.length > 0;
+}
diff --git a/common/lib/settingsDocs.ts b/common/lib/settingsDocs.ts
@@ -21,6 +21,7 @@ import {
PUBLISH_SETTINGS_FIELD_DOCS,
DIGEST_SETTINGS_FIELD_DOCS,
SAVED_VIDEO_BACKUP_SETTINGS_FIELD_DOCS,
+ SEEDER_SETTINGS_FIELD_DOCS,
SOCIAL_LINK_FIELD_DOCS,
SOCIAL_SETTINGS_FIELD_DOCS,
X_SOCIAL_SETTINGS_FIELD_DOCS,
@@ -252,6 +253,13 @@ export function blockTables(d: SiteSettings): Partial<Record<keyof SiteSettings,
defaults: fromObject(d.publish),
},
],
+ seeder: [
+ {
+ path: "seeder",
+ docs: SEEDER_SETTINGS_FIELD_DOCS,
+ defaults: fromObject(d.seeder),
+ },
+ ],
};
}
diff --git a/common/lib/settingsSchema.test.ts b/common/lib/settingsSchema.test.ts
@@ -30,6 +30,7 @@ import type {
} from "../ytdlp/downloadFormat";
import type { StorageSettings } from "./storageLocations";
import type { ArchiveOrgFetchSettings } from "./archiveOrgTorrent";
+import type { SeederSettings } from "./seederSettings";
import type {
AttributionSettings,
BackfillSettings,
@@ -103,13 +104,15 @@ type PreSchemaSiteSettings = {
archiveOrg: ArchiveOrgFetchSettings;
// The publish lane (release 18).
publish: PublishSettings;
+ // The home seeder of last resort (release 21 D4b).
+ seeder: SeederSettings;
};
// Bracketed so the conditional does not distribute (see commit 8c43231).
type Same<A, B> = [A] extends [B] ? ([B] extends [A] ? true : false) : false;
const shapeUnchanged: Same<SiteSettings, PreSchemaSiteSettings> = true;
-test("SiteSettings keeps its 36 fields, in file order", () => {
+test("SiteSettings keeps its 37 fields, in file order", () => {
assert.equal(shapeUnchanged, true);
assert.deepEqual(Object.keys(siteSettingsSchema.shape), [
"adminTitle",
@@ -148,6 +151,7 @@ test("SiteSettings keeps its 36 fields, in file order", () => {
"attribution",
"archiveOrg",
"publish",
+ "seeder",
]);
// A parsed object carries every key, in that order — writeSettings writes
// exactly this, so the order is the on-disk order.
diff --git a/common/lib/settingsSchema.ts b/common/lib/settingsSchema.ts
@@ -76,6 +76,8 @@ import {
DEFAULT_ARCHIVE_ORG_FETCH_SETTINGS,
type ArchiveOrgFetchSettings,
} from "./archiveOrgTorrent";
+import { sanitizeSeeder, type SeederSettings } from "./seederSettings";
+export { SEEDER_SETTINGS_FIELD_DOCS, type SeederSettings } from "./seederSettings";
import {
DEFAULT_COOKIE_MODE,
isCookieMode,
@@ -1822,6 +1824,9 @@ export const siteSettingsSchema = z.object({
publish: settingsField((v): PublishSettings => sanitizePublish(v)).describe(
"The publish LANE (release 18): a runner that, when the index is stale, updates it and then builds — and, where a site's own `publish.auto` (site.json) says so, deploys — what changed, one stage at a time on the `publish` queue. OFF by default; the manual stages (Publish now, Build, Deploy) work either way. See PublishSettings and common/publish/publishRunner.ts.",
),
+ seeder: settingsField((v): SeederSettings => sanitizeSeeder(v)).describe(
+ "The home seeder of last resort (release 21): `archilyzer seed` seeds the sites' playable torrents (`archilyzer media playable`), each one only while no other seeder has it, and only behind a VPN (the docker profile `seeder`, docker-compose.seeder.yml). Not configured until `sites` names one. See SeederSettings and common/controller/seeder.ts.",
+ ),
});
export type SiteSettings = z.infer<typeof siteSettingsSchema>;
diff --git a/common/lib/zipReader.test.ts b/common/lib/zipReader.test.ts
@@ -0,0 +1,176 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import { createHash } from "node:crypto";
+import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
+import { tmpdir } from "node:os";
+import path from "node:path";
+import { execa } from "execa";
+import { makeZip } from "../testing/makeZip";
+import {
+ ZipFormatError,
+ copyFileHashed,
+ copyZipEntry,
+ readZipEntries,
+ readZipEntryText,
+ zipEntryProblem,
+} from "./zipReader";
+
+// Run with: pnpm --filter yt-dlp-transcript-common exec tsx --test lib/zipReader.test.ts
+
+const sha = (b: Buffer) => createHash("sha256").update(b).digest("hex");
+const VIDEO = Buffer.from(Array.from({ length: 200_000 }, (_, i) => (i * 7) & 0xff));
+
+async function withTmp(fn: (dir: string) => Promise<void>): Promise<void> {
+ const dir = await mkdtemp(path.join(tmpdir(), "ttb-zip-"));
+ try {
+ await fn(dir);
+ } finally {
+ await rm(dir, { recursive: true, force: true });
+ }
+}
+
+for (const zip64 of [false, true]) {
+ test(`reads the central directory${zip64 ? " (ZIP64)" : ""}: names, sizes, methods, directories`, async () => {
+ await withTmp(async (dir) => {
+ const file = path.join(dir, "a.zip");
+ await writeFile(
+ file,
+ makeZip(
+ [
+ { name: "YouTube/" },
+ { name: "YouTube/001 - Ünïcode ｜ name - (abcdefghijk)/v-abcdefghijk.mp4", data: VIDEO },
+ { name: "YouTube/001 - Ünïcode ｜ name - (abcdefghijk)/description.txt", data: "hello", deflate: true },
+ ],
+ { zip64 },
+ ),
+ );
+ const entries = await readZipEntries(file);
+ assert.equal(entries.length, 3);
+ assert.equal(entries[0].isDirectory, true);
+ assert.equal(entries[1].name, "YouTube/001 - Ünïcode ｜ name - (abcdefghijk)/v-abcdefghijk.mp4");
+ assert.equal(entries[1].method, 0);
+ assert.equal(entries[1].size, VIDEO.length);
+ assert.equal(entries[2].method, 8);
+ assert.equal(entries[2].size, 5);
+ });
+ });
+
+ test(`copies a stored and a deflated entry out, hashed and CRC-checked${zip64 ? " (ZIP64)" : ""}`, async () => {
+ await withTmp(async (dir) => {
+ const file = path.join(dir, "a.zip");
+ await writeFile(
+ file,
+ makeZip(
+ [
+ { name: "a/stored.mp4", data: VIDEO },
+ { name: "a/deflated.mkv", data: VIDEO, deflate: true },
+ { name: "a/empty.txt", data: "" },
+ ],
+ { zip64 },
+ ),
+ );
+ const entries = await readZipEntries(file);
+ for (const [i, name] of [[0, "s"], [1, "d"]] as const) {
+ const dest = path.join(dir, name);
+ const got = await copyZipEntry(file, entries[i], dest);
+ assert.equal(got.bytes, VIDEO.length);
+ assert.equal(got.sha256, sha(VIDEO));
+ assert.deepEqual(await readFile(dest), VIDEO);
+ }
+ const empty = await copyZipEntry(file, entries[2], path.join(dir, "e"));
+ assert.equal(empty.bytes, 0);
+ });
+ });
+}
+
+test("a damaged entry is refused by its CRC, and the archive is never written", async () => {
+ await withTmp(async (dir) => {
+ const file = path.join(dir, "a.zip");
+ const zip = makeZip([{ name: "x.mp4", data: VIDEO }]);
+ // Flip one byte of the stored data (the local header is 30 + name bytes).
+ zip[30 + "x.mp4".length + 100] ^= 0xff;
+ await writeFile(file, zip);
+ const before = sha(await readFile(file));
+ const [entry] = await readZipEntries(file);
+ await assert.rejects(copyZipEntry(file, entry, path.join(dir, "out")), /CRC32 mismatch/);
+ assert.equal(sha(await readFile(file)), before);
+ });
+});
+
+test("an existing destination is never overwritten (staging files are exclusive)", async () => {
+ await withTmp(async (dir) => {
+ const file = path.join(dir, "a.zip");
+ await writeFile(file, makeZip([{ name: "x.mp4", data: VIDEO }]));
+ const [entry] = await readZipEntries(file);
+ await writeFile(path.join(dir, "out"), "keep");
+ await assert.rejects(copyZipEntry(file, entry, path.join(dir, "out")));
+ assert.equal(await readFile(path.join(dir, "out"), "utf8"), "keep");
+ });
+});
+
+test("text entries read, stored or deflated; a big one is refused", async () => {
+ await withTmp(async (dir) => {
+ const file = path.join(dir, "a.zip");
+ await writeFile(
+ file,
+ makeZip([
+ { name: "a.info.json", data: '{"id":"x"}' },
+ { name: "description.txt", data: "Published on July 13th 2018", deflate: true },
+ { name: "big.mp4", data: VIDEO },
+ ]),
+ );
+ const [info, desc, big] = await readZipEntries(file);
+ assert.equal(await readZipEntryText(file, info), '{"id":"x"}');
+ assert.equal(await readZipEntryText(file, desc), "Published on July 13th 2018");
+ await assert.rejects(readZipEntryText(file, big, 1000), /too large/);
+ });
+});
+
+test("unsupported methods and encryption are named, not half-read", () => {
+ const base = { name: "x", isDirectory: false, compressedSize: 1, size: 1, crc32: 0, localHeaderOffset: 0, encrypted: false };
+ assert.equal(zipEntryProblem({ ...base, method: 0 }), null);
+ assert.equal(zipEntryProblem({ ...base, method: 8 }), null);
+ assert.match(zipEntryProblem({ ...base, method: 14 }) ?? "", /method 14/);
+ assert.equal(zipEntryProblem({ ...base, method: 0, encrypted: true }), "encrypted");
+});
+
+test("not a zip: a sentence, not a crash", async () => {
+ await withTmp(async (dir) => {
+ const file = path.join(dir, "not.zip");
+ await writeFile(file, "plain text, no end record");
+ await assert.rejects(readZipEntries(file), ZipFormatError);
+ });
+});
+
+test("copyFileHashed hashes a plain file the same way", async () => {
+ await withTmp(async (dir) => {
+ await writeFile(path.join(dir, "src"), VIDEO);
+ const got = await copyFileHashed(path.join(dir, "src"), path.join(dir, "dst"));
+ assert.equal(got.sha256, sha(VIDEO));
+ assert.deepEqual(await readFile(path.join(dir, "dst")), VIDEO);
+ });
+});
+
+// Interop with a real writer: Info-ZIP's `zip`, stored (-0) and deflated, and
+// forced ZIP64 (-fz). Skipped where `zip` is not installed.
+test("reads what Info-ZIP's zip writes, ZIP64 included", async (t) => {
+ const probe = await execa("zip", ["-v"], { reject: false }).catch(() => null);
+ if (!probe || probe.exitCode !== 0) {
+ t.skip("zip is not installed");
+ return;
+ }
+ await withTmp(async (dir) => {
+ await writeFile(path.join(dir, "v.mp4"), VIDEO);
+ await writeFile(path.join(dir, "d.txt"), "some text ".repeat(100));
+ for (const args of [["-0"], ["-9"], ["-0", "-fz"]]) {
+ const file = path.join(dir, `out${args.join("")}.zip`);
+ await execa("zip", ["-q", "-X", ...args, file, "v.mp4", "d.txt"], { cwd: dir });
+ const entries = await readZipEntries(file);
+ const v = entries.find((e) => e.name === "v.mp4")!;
+ const got = await copyZipEntry(file, v, path.join(dir, `copy${args.join("")}`));
+ assert.equal(got.sha256, sha(VIDEO), args.join(" "));
+ const d = entries.find((e) => e.name === "d.txt")!;
+ assert.equal(await readZipEntryText(file, d), "some text ".repeat(100));
+ }
+ });
+});
diff --git a/common/lib/zipReader.ts b/common/lib/zipReader.ts
@@ -0,0 +1,308 @@
+// A ZIP READ IN PLACE (release 21 D1, controller/attachMedia.ts).
+//
+// The archives media is attached from are tens of GB and mostly STORED (no
+// compression: video does not deflate), so the useful operation is "copy this
+// entry's bytes out", and for a stored entry that is a byte range of the zip
+// file itself — no temp dir, no extraction tool, no second copy. A deflated
+// entry is inflated on the way out. Anything else (bzip2, LZMA, encryption) is
+// refused by name rather than half-read.
+//
+// Only the CENTRAL DIRECTORY is trusted for sizes and offsets (the local
+// header's sizes are zero when bit 3 is set), with ZIP64 throughout: a 36 GB
+// archive's later offsets do not fit 32 bits. The local header is read only
+// for its own name/extra lengths, which is where the data starts.
+//
+// READ-ONLY. Every open here is `r`; nothing writes to the archive.
+//
+// SERVER-ONLY (node:fs).
+
+import { createReadStream, createWriteStream } from "node:fs";
+import { open, type FileHandle } from "node:fs/promises";
+import { createHash } from "node:crypto";
+import { Transform, type TransformCallback } from "node:stream";
+import { pipeline } from "node:stream/promises";
+import { createInflateRaw, crc32, inflateRawSync } from "node:zlib";
+import { Readable } from "node:stream";
+
+export type ZipEntry = {
+ // The entry's path inside the archive, "/"-separated, as stored.
+ name: string;
+ isDirectory: boolean;
+ // 0 = stored, 8 = deflated; anything else is refused at extraction.
+ method: number;
+ compressedSize: number;
+ size: number;
+ crc32: number;
+ localHeaderOffset: number;
+ encrypted: boolean;
+};
+
+const EOCD_SIG = 0x06054b50;
+const ZIP64_LOCATOR_SIG = 0x07064b50;
+const ZIP64_EOCD_SIG = 0x06064b50;
+const CDH_SIG = 0x02014b50;
+const LFH_SIG = 0x04034b50;
+const U32_MAX = 0xffffffff;
+const U16_MAX = 0xffff;
+
+export class ZipFormatError extends Error {}
+
+async function readAt(fh: FileHandle, offset: number, length: number): Promise<Buffer> {
+ const buf = Buffer.alloc(length);
+ let got = 0;
+ while (got < length) {
+ const { bytesRead } = await fh.read(buf, got, length - got, offset + got);
+ if (bytesRead === 0) break;
+ got += bytesRead;
+ }
+ return got === length ? buf : buf.subarray(0, got);
+}
+
+// A 64-bit little-endian value as a JS number. Every offset and size in a
+// real archive is far below 2^53; one that is not is a corrupt file.
+function u64(buf: Buffer, at: number): number {
+ const v = buf.readBigUInt64LE(at);
+ if (v > BigInt(Number.MAX_SAFE_INTEGER)) {
+ throw new ZipFormatError("a ZIP64 value exceeds 2^53");
+ }
+ return Number(v);
+}
+
+const utf8Strict = new TextDecoder("utf-8", { fatal: true });
+
+// Bit 11 says UTF-8. Without it the spec says CP437, but in practice archives
+// written on Linux and by most tools are UTF-8 without the flag; valid UTF-8
+// is read as UTF-8, anything else as latin1 (identical to CP437 for ASCII).
+function decodeName(raw: Buffer, utf8Flag: boolean): string {
+ if (utf8Flag) return raw.toString("utf8");
+ try {
+ return utf8Strict.decode(raw);
+ } catch {
+ return raw.toString("latin1");
+ }
+}
+
+type Extra = { id: number; data: Buffer };
+
+function parseExtras(buf: Buffer): Extra[] {
+ const out: Extra[] = [];
+ let at = 0;
+ while (at + 4 <= buf.length) {
+ const id = buf.readUInt16LE(at);
+ const len = buf.readUInt16LE(at + 2);
+ const data = buf.subarray(at + 4, at + 4 + len);
+ out.push({ id, data });
+ at += 4 + len;
+ }
+ return out;
+}
+
+async function findEocd(fh: FileHandle, fileSize: number): Promise<{ buf: Buffer; at: number }> {
+ // EOCD is 22 bytes plus a comment of up to 65535.
+ const tail = Math.min(fileSize, 22 + 0xffff);
+ const buf = await readAt(fh, fileSize - tail, tail);
+ for (let i = buf.length - 22; i >= 0; i--) {
+ if (buf.readUInt32LE(i) === EOCD_SIG) {
+ return { buf: buf.subarray(i), at: fileSize - tail + i };
+ }
+ }
+ throw new ZipFormatError("not a zip file (no end-of-central-directory record)");
+}
+
+// Every entry of the archive, from its central directory.
+export async function readZipEntries(file: string): Promise<ZipEntry[]> {
+ const fh = await open(file, "r");
+ try {
+ const { size: fileSize } = await fh.stat();
+ const eocd = await findEocd(fh, fileSize);
+ let count = eocd.buf.readUInt16LE(10);
+ let cdSize = eocd.buf.readUInt32LE(12);
+ let cdOffset = eocd.buf.readUInt32LE(16);
+ if (count === U16_MAX || cdSize === U32_MAX || cdOffset === U32_MAX) {
+ if (eocd.at < 20) throw new ZipFormatError("ZIP64 locator missing");
+ const loc = await readAt(fh, eocd.at - 20, 20);
+ if (loc.readUInt32LE(0) !== ZIP64_LOCATOR_SIG) {
+ throw new ZipFormatError("ZIP64 locator missing");
+ }
+ const z64At = u64(loc, 8);
+ const z64 = await readAt(fh, z64At, 56);
+ if (z64.readUInt32LE(0) !== ZIP64_EOCD_SIG) {
+ throw new ZipFormatError("ZIP64 end-of-central-directory record missing");
+ }
+ count = u64(z64, 32);
+ cdSize = u64(z64, 40);
+ cdOffset = u64(z64, 48);
+ }
+ const cd = await readAt(fh, cdOffset, cdSize);
+ const entries: ZipEntry[] = [];
+ let at = 0;
+ for (let i = 0; i < count; i++) {
+ if (at + 46 > cd.length || cd.readUInt32LE(at) !== CDH_SIG) {
+ throw new ZipFormatError(`central directory entry ${i} is malformed`);
+ }
+ const flags = cd.readUInt16LE(at + 8);
+ const method = cd.readUInt16LE(at + 10);
+ const crc = cd.readUInt32LE(at + 16);
+ let compressedSize = cd.readUInt32LE(at + 20);
+ let size = cd.readUInt32LE(at + 24);
+ const nameLen = cd.readUInt16LE(at + 28);
+ const extraLen = cd.readUInt16LE(at + 30);
+ const commentLen = cd.readUInt16LE(at + 32);
+ let localHeaderOffset = cd.readUInt32LE(at + 42);
+ const rawName = cd.subarray(at + 46, at + 46 + nameLen);
+ const extras = parseExtras(cd.subarray(at + 46 + nameLen, at + 46 + nameLen + extraLen));
+ let name = decodeName(rawName, (flags & 0x800) !== 0);
+ for (const ex of extras) {
+ if (ex.id === 0x0001) {
+ // ZIP64: only the fields whose 32-bit slot is saturated, in order.
+ let p = 0;
+ if (size === U32_MAX) { size = u64(ex.data, p); p += 8; }
+ if (compressedSize === U32_MAX) { compressedSize = u64(ex.data, p); p += 8; }
+ if (localHeaderOffset === U32_MAX) { localHeaderOffset = u64(ex.data, p); p += 8; }
+ } else if (ex.id === 0x7075 && ex.data.length > 5 && ex.data[0] === 1) {
+ // Info-ZIP Unicode Path: version 1, CRC32 of the raw name, UTF-8 name.
+ if (ex.data.readUInt32LE(1) === crc32(rawName)) {
+ name = ex.data.subarray(5).toString("utf8");
+ }
+ }
+ }
+ name = name.replace(/\\/g, "/");
+ entries.push({
+ name,
+ isDirectory: name.endsWith("/"),
+ method,
+ compressedSize,
+ size,
+ crc32: crc,
+ localHeaderOffset,
+ encrypted: (flags & 0x1) !== 0,
+ });
+ at += 46 + nameLen + extraLen + commentLen;
+ }
+ return entries;
+ } finally {
+ await fh.close();
+ }
+}
+
+// Where an entry's data begins: past its local header, whose name and extra
+// lengths may differ from the central directory's.
+export async function zipEntryDataOffset(file: string, entry: ZipEntry): Promise<number> {
+ const fh = await open(file, "r");
+ try {
+ const lfh = await readAt(fh, entry.localHeaderOffset, 30);
+ if (lfh.length < 30 || lfh.readUInt32LE(0) !== LFH_SIG) {
+ throw new ZipFormatError(`no local header for ${entry.name}`);
+ }
+ return entry.localHeaderOffset + 30 + lfh.readUInt16LE(26) + lfh.readUInt16LE(28);
+ } finally {
+ await fh.close();
+ }
+}
+
+// Why an entry cannot be copied out, or null when it can.
+export function zipEntryProblem(entry: ZipEntry): string | null {
+ if (entry.isDirectory) return "a directory";
+ if (entry.encrypted) return "encrypted";
+ if (entry.method !== 0 && entry.method !== 8) {
+ return `compression method ${entry.method} (only stored and deflated are read)`;
+ }
+ return null;
+}
+
+// Counts bytes, CRC32 and sha256 of what passes through it.
+class Digest extends Transform {
+ bytes = 0;
+ crc = 0;
+ readonly hash = createHash("sha256");
+ _transform(chunk: Buffer, _enc: BufferEncoding, cb: TransformCallback): void {
+ this.bytes += chunk.length;
+ this.crc = crc32(chunk, this.crc);
+ this.hash.update(chunk);
+ cb(null, chunk);
+ }
+}
+
+export type CopiedEntry = { bytes: number; sha256: string };
+
+// Copy one entry's (uncompressed) bytes to `dest`, verifying its size and
+// CRC32 against the central directory. A stored entry is a byte range of the
+// archive; a deflated one is inflated. `dest` is written and left in place on
+// success; on any failure it is removed by the caller (it is a staging path).
+export async function copyZipEntry(
+ file: string,
+ entry: ZipEntry,
+ dest: string,
+ opts: { signal?: AbortSignal } = {},
+): Promise<CopiedEntry> {
+ const problem = zipEntryProblem(entry);
+ if (problem) throw new ZipFormatError(`${entry.name}: ${problem}`);
+ const start = await zipEntryDataOffset(file, entry);
+ const digest = new Digest();
+ const out = createWriteStream(dest, { flags: "wx" });
+ if (entry.compressedSize === 0) {
+ await pipeline(Readable.from([]), digest, out, { signal: opts.signal });
+ } else {
+ const src = createReadStream(file, {
+ start,
+ end: start + entry.compressedSize - 1,
+ highWaterMark: 1 << 20,
+ });
+ if (entry.method === 0) {
+ await pipeline(src, digest, out, { signal: opts.signal });
+ } else {
+ await pipeline(src, createInflateRaw(), digest, out, { signal: opts.signal });
+ }
+ }
+ if (digest.bytes !== entry.size) {
+ throw new ZipFormatError(
+ `${entry.name}: ${digest.bytes} bytes copied, the archive says ${entry.size}`,
+ );
+ }
+ if (digest.crc >>> 0 !== entry.crc32 >>> 0) {
+ throw new ZipFormatError(`${entry.name}: CRC32 mismatch — the archive is damaged here`);
+ }
+ return { bytes: digest.bytes, sha256: digest.hash.digest("hex") };
+}
+
+// A small entry's text (an .info.json, a description.txt). Refuses anything
+// over `maxBytes` so a mislabelled video is never read into memory.
+export async function readZipEntryText(
+ file: string,
+ entry: ZipEntry,
+ maxBytes = 4 * 1024 * 1024,
+): Promise<string> {
+ const problem = zipEntryProblem(entry);
+ if (problem) throw new ZipFormatError(`${entry.name}: ${problem}`);
+ if (entry.size > maxBytes) {
+ throw new ZipFormatError(`${entry.name}: ${entry.size} bytes is too large to read as text`);
+ }
+ const start = await zipEntryDataOffset(file, entry);
+ const fh = await open(file, "r");
+ let raw: Buffer;
+ try {
+ raw = await readAt(fh, start, entry.compressedSize);
+ } finally {
+ await fh.close();
+ }
+ if (entry.method === 8) {
+ raw = inflateRawSync(raw);
+ }
+ return raw.toString("utf8");
+}
+
+// Hash a plain file the same way (a directory source's entry).
+export async function copyFileHashed(
+ src: string,
+ dest: string,
+ opts: { signal?: AbortSignal } = {},
+): Promise<CopiedEntry> {
+ const digest = new Digest();
+ await pipeline(
+ createReadStream(src, { highWaterMark: 1 << 20 }),
+ digest,
+ createWriteStream(dest, { flags: "wx" }),
+ { signal: opts.signal },
+ );
+ return { bytes: digest.bytes, sha256: digest.hash.digest("hex") };
+}
diff --git a/common/package.json b/common/package.json
@@ -58,9 +58,11 @@
"@sindresorhus/slugify": "^3.0.0",
"@tanstack/react-query": "^5.99.1",
"@tanstack/react-virtual": "^3.13.0",
+ "bittorrent-tracker": "^11.2.3",
"class-variance-authority": "^0.7.1",
"clsx": "^2.1.1",
"cmdk": "^1.1.1",
+ "create-torrent": "^6.1.3",
"d3-scale": "^4.0.2",
"execa": "^9.6.1",
"flexsearch": "^0.8.0",
@@ -70,12 +72,14 @@
"lucide-react": "^1.16.0",
"markdown-to-jsx": "^7.7.4",
"p-limit": "^7.3.0",
+ "parse-torrent": "^11.0.24",
"radix-ui": "^1.6.0",
"react-player": "^2.16.1",
"recharts": "2.15.4",
"sonner": "^2.0.7",
"tailwind-merge": "^3.6.0",
"tw-animate-css": "^1.4.0",
+ "webtorrent": "^3.0.21",
"zod": "^4.3.6"
},
"peerDependencies": {
diff --git a/common/testing/makeZip.ts b/common/testing/makeZip.ts
@@ -0,0 +1,95 @@
+// A TINY ZIP WRITER, for the unit tests of lib/zipReader.ts and
+// controller/attachMedia.ts: stored and deflated entries, UTF-8 names, and a
+// forced-ZIP64 variant (every size and offset in the ZIP64 extra field, the
+// ZIP64 end records present), so the reader's 64-bit paths run without a
+// 4 GB fixture. Synthetic bytes only — never a real archive.
+
+import { crc32, deflateRawSync } from "node:zlib";
+
+export type ZipFixtureEntry = {
+ name: string;
+ data?: Buffer | string;
+ // Default: stored. A directory entry has a name ending in "/" and no data.
+ deflate?: boolean;
+};
+
+export function makeZip(entries: ZipFixtureEntry[], opts: { zip64?: boolean } = {}): Buffer {
+ const zip64 = !!opts.zip64;
+ const locals: Buffer[] = [];
+ const centrals: Buffer[] = [];
+ let offset = 0;
+ for (const e of entries) {
+ const name = Buffer.from(e.name, "utf8");
+ const raw = e.data === undefined ? Buffer.alloc(0) : Buffer.isBuffer(e.data) ? e.data : Buffer.from(e.data, "utf8");
+ const method = e.deflate ? 8 : 0;
+ const body = e.deflate ? deflateRawSync(raw) : raw;
+ const crc = crc32(raw) >>> 0;
+ const flags = 0x800;
+
+ const localExtra = zip64 ? z64Extra([raw.length, body.length]) : Buffer.alloc(0);
+ const lfh = Buffer.alloc(30);
+ lfh.writeUInt32LE(0x04034b50, 0);
+ lfh.writeUInt16LE(zip64 ? 45 : 20, 4);
+ lfh.writeUInt16LE(flags, 6);
+ lfh.writeUInt16LE(method, 8);
+ lfh.writeUInt32LE(crc, 14);
+ lfh.writeUInt32LE(zip64 ? 0xffffffff : body.length, 18);
+ lfh.writeUInt32LE(zip64 ? 0xffffffff : raw.length, 22);
+ lfh.writeUInt16LE(name.length, 26);
+ lfh.writeUInt16LE(localExtra.length, 28);
+ const local = Buffer.concat([lfh, name, localExtra, body]);
+
+ const cdExtra = zip64 ? z64Extra([raw.length, body.length, offset]) : Buffer.alloc(0);
+ const cdh = Buffer.alloc(46);
+ cdh.writeUInt32LE(0x02014b50, 0);
+ cdh.writeUInt16LE(zip64 ? 45 : 20, 4);
+ cdh.writeUInt16LE(zip64 ? 45 : 20, 6);
+ cdh.writeUInt16LE(flags, 8);
+ cdh.writeUInt16LE(method, 10);
+ cdh.writeUInt32LE(crc, 16);
+ cdh.writeUInt32LE(zip64 ? 0xffffffff : body.length, 20);
+ cdh.writeUInt32LE(zip64 ? 0xffffffff : raw.length, 24);
+ cdh.writeUInt16LE(name.length, 28);
+ cdh.writeUInt16LE(cdExtra.length, 30);
+ cdh.writeUInt32LE(zip64 ? 0xffffffff : offset, 42);
+ centrals.push(Buffer.concat([cdh, name, cdExtra]));
+
+ locals.push(local);
+ offset += local.length;
+ }
+ const cd = Buffer.concat(centrals);
+ const cdOffset = offset;
+ const tail: Buffer[] = [];
+ if (zip64) {
+ const rec = Buffer.alloc(56);
+ rec.writeUInt32LE(0x06064b50, 0);
+ rec.writeBigUInt64LE(44n, 4);
+ rec.writeUInt16LE(45, 12);
+ rec.writeUInt16LE(45, 14);
+ rec.writeBigUInt64LE(BigInt(entries.length), 24);
+ rec.writeBigUInt64LE(BigInt(entries.length), 32);
+ rec.writeBigUInt64LE(BigInt(cd.length), 40);
+ rec.writeBigUInt64LE(BigInt(cdOffset), 48);
+ const loc = Buffer.alloc(20);
+ loc.writeUInt32LE(0x07064b50, 0);
+ loc.writeBigUInt64LE(BigInt(cdOffset + cd.length), 8);
+ loc.writeUInt32LE(1, 16);
+ tail.push(rec, loc);
+ }
+ const eocd = Buffer.alloc(22);
+ eocd.writeUInt32LE(0x06054b50, 0);
+ eocd.writeUInt16LE(zip64 ? 0xffff : entries.length, 8);
+ eocd.writeUInt16LE(zip64 ? 0xffff : entries.length, 10);
+ eocd.writeUInt32LE(zip64 ? 0xffffffff : cd.length, 12);
+ eocd.writeUInt32LE(zip64 ? 0xffffffff : cdOffset, 16);
+ tail.push(eocd);
+ return Buffer.concat([...locals, cd, ...tail]);
+}
+
+function z64Extra(values: number[]): Buffer {
+ const b = Buffer.alloc(4 + 8 * values.length);
+ b.writeUInt16LE(0x0001, 0);
+ b.writeUInt16LE(8 * values.length, 2);
+ values.forEach((v, i) => b.writeBigUInt64LE(BigInt(v), 4 + 8 * i));
+ return b;
+}
diff --git a/common/types/webtorrent-modules.d.ts b/common/types/webtorrent-modules.d.ts
@@ -0,0 +1,128 @@
+// Types for the WebTorrent-ecosystem packages release 21 uses, which ship
+// none: only the surface the code calls.
+
+declare module "create-torrent" {
+ export type CreateTorrentOptions = {
+ name?: string;
+ comment?: string;
+ createdBy?: string;
+ creationDate?: Date | number;
+ private?: boolean;
+ pieceLength?: number;
+ announceList?: string[][];
+ urlList?: string[];
+ info?: Record<string, unknown>;
+ };
+ export default function createTorrent(
+ input: string | string[] | Uint8Array,
+ opts: CreateTorrentOptions,
+ cb: (err: Error | null, torrent: Uint8Array) => void,
+ ): void;
+}
+
+declare module "webtorrent" {
+ import type { EventEmitter } from "node:events";
+ export type WebTorrentOptions = {
+ maxConns?: number;
+ uploadLimit?: number;
+ downloadLimit?: number;
+ dht?: boolean | Record<string, unknown>;
+ lsd?: boolean;
+ natUpnp?: boolean;
+ natPmp?: boolean;
+ utp?: boolean;
+ utPex?: boolean;
+ webSeeds?: boolean;
+ tracker?: boolean | Record<string, unknown>;
+ torrentPort?: number;
+ };
+ export type AddOptions = {
+ path?: string;
+ announce?: string[];
+ skipVerify?: boolean;
+ private?: boolean;
+ };
+ export interface Wire {
+ isSeeder: boolean;
+ peerId?: string;
+ remoteAddress?: string;
+ type?: string;
+ }
+ export interface Torrent extends EventEmitter {
+ infoHash: string;
+ name: string;
+ ready: boolean;
+ done: boolean;
+ progress: number;
+ downloaded: number;
+ uploaded: number;
+ numPeers: number;
+ wires: Wire[];
+ destroyed: boolean;
+ destroy(opts?: { destroyStore?: boolean } | null, cb?: (err?: Error | null) => void): void;
+ }
+ export default class WebTorrent extends EventEmitter {
+ static WEBRTC_SUPPORT: boolean;
+ static VERSION: string;
+ constructor(opts?: WebTorrentOptions);
+ torrents: Torrent[];
+ torrentPort: number;
+ destroyed: boolean;
+ add(torrentId: Uint8Array | string, opts?: AddOptions, onTorrent?: (t: Torrent) => void): Torrent;
+ seed(input: string | Uint8Array, opts?: Record<string, unknown>, onSeed?: (t: Torrent) => void): Torrent;
+ remove(torrentId: Torrent | string, opts?: { destroyStore?: boolean } | null, cb?: (err?: Error | null) => void): Promise<void>;
+ throttleUpload(rate: number): void;
+ destroy(cb?: (err?: Error | null) => void): void;
+ address(): { address: string; family: string; port: number } | null;
+ }
+}
+
+declare module "bittorrent-tracker" {
+ import type { EventEmitter } from "node:events";
+ export type ScrapeData = { announce: string; infoHash: string; complete: number; incomplete: number; downloaded: number };
+ export class Client extends EventEmitter {
+ static scrape(
+ opts: { announce: string; infoHash: string | string[] },
+ cb: (err: Error | null, data: ScrapeData | Record<string, ScrapeData>) => void,
+ ): Client;
+ destroy(cb?: () => void): void;
+ }
+ export type ServerOptions = {
+ http?: boolean;
+ udp?: boolean;
+ ws?: boolean;
+ stats?: boolean;
+ trustProxy?: boolean;
+ interval?: number;
+ filter?: (infoHash: string, params: unknown, cb: (err?: Error | null) => void) => void;
+ };
+ export class Server extends EventEmitter {
+ constructor(opts?: ServerOptions);
+ torrents: Record<string, { complete: number; incomplete: number; peers: unknown }>;
+ listen(port: number, hostname: string, cb?: () => void): void;
+ address(): { address: string; port: number } | null;
+ http: { address(): { address: string; port: number } | null } | null;
+ close(cb?: () => void): void;
+ }
+ export default Client;
+}
+
+declare module "parse-torrent" {
+ export type ParsedTorrent = {
+ infoHash: string;
+ name?: string;
+ length?: number;
+ pieceLength?: number;
+ announce?: string[];
+ urlList?: string[];
+ created?: Date;
+ createdBy?: string;
+ comment?: string;
+ private?: boolean;
+ info?: Record<string, unknown>;
+ files?: { path: string; name: string; length: number; offset: number }[];
+ };
+ export default function parseTorrent(torrent: Uint8Array | string): Promise<ParsedTorrent>;
+ export function toTorrentFile(parsed: ParsedTorrent): Uint8Array;
+ export function toMagnetURI(parsed: Partial<ParsedTorrent>): string;
+}
diff --git a/docker-compose.seeder.yml b/docker-compose.seeder.yml
@@ -0,0 +1,108 @@
+# The home seeder, behind a VPN, with no way around it (release 21 D4b).
+#
+# SEEDER_WG_CONF=/abs/path/to/wg0.conf \
+# docker compose -f docker-compose.yml -f docker-compose.seeder.yml --profile seeder up -d
+#
+# `archilyzer seed` is the peer of last resort for the sites' playable
+# torrents (settings.seeder; SETTINGS.md): it seeds a torrent only while no
+# other seeder has it. Every byte it uploads leaves through the tunnel below.
+#
+# THE KILL SWITCH IS THE CONSTRUCTION, not a setting:
+#
+# * `seeder-vpn` is gluetun: a WireGuard client whose firewall drops every
+# packet that is not in the tunnel (or to the tunnel's own endpoint). Tunnel
+# down = no network, never the home connection.
+# * `seeder` and `tracker` have NO network of their own: `network_mode:
+# service:seeder-vpn` puts them in that container's network namespace, so
+# the only interfaces they can see are the tunnel and loopback (plus
+# gluetun's own, which its firewall closes). WebRTC gathers ICE candidates
+# from what it can see, so a browser peer is never offered the home address.
+# * Neither publishes a port. Inbound connections (a browser or a desktop
+# client dialing the seeder, a browser reaching the tracker) need the VPN
+# provider's port forwarding — gluetun's FIREWALL_VPN_INPUT_PORTS. Whether
+# to rely on that, or on a TURN relay, is the operator's ruling (the plan's
+# "risk to measure first"); nothing here opens one by default.
+# * `seeder.bindInterface` in settings.json (e.g. "tun0" or "wg0", whatever
+# gluetun names it) makes `archilyzer seed` refuse to start, and stop every
+# torrent, while that interface is gone — the same rule, a second time, in
+# the app.
+#
+# THE WIREGUARD CONFIG IS YOURS. `SEEDER_WG_CONF` is the absolute path of a
+# wg-quick file from your VPN provider (an [Interface] with your PrivateKey and
+# Address, a [Peer] with the endpoint). There is no default and nothing here
+# invents one: compose refuses to start the profile without it. It is mounted
+# read-only and never copied into an image.
+#
+# `archilyzer doctor` compares the seeder's egress IP with the host's and fails
+# if they match ("seeder" section); it says "seeder not configured" while
+# settings.seeder.sites is empty.
+
+services:
+ seeder-vpn:
+ profiles: [seeder]
+ # Pinned. gluetun's firewall — always on in v3, there is no switch for it —
+ # is what makes the tunnel the only way out of this namespace.
+ image: qmcgaw/gluetun:v3.41.3
+ restart: unless-stopped
+ cap_add: [NET_ADMIN]
+ devices:
+ - /dev/net/tun:/dev/net/tun
+ environment:
+ VPN_SERVICE_PROVIDER: custom
+ VPN_TYPE: wireguard
+ WIREGUARD_CONF_SECRETFILE: /gluetun/wireguard/wg0.conf
+ # Inbound through the tunnel, when the provider forwards a port to you:
+ # the seeder's torrent port and the tracker's. Empty = none.
+ FIREWALL_VPN_INPUT_PORTS: ${SEEDER_VPN_INPUT_PORTS:-}
+ TZ: ${TZ:-UTC}
+ volumes:
+ - ${SEEDER_WG_CONF:?set SEEDER_WG_CONF to your VPN provider's wg-quick file}:/gluetun/wireguard/wg0.conf:ro
+
+ # The app image (docker-compose.yml's), run as `archilyzer seed`. YAML
+ # anchors do not cross files, so the few keys it needs are spelled here.
+ seeder:
+ image: archilyzer:${ARCHILYZER_TAG:-local}
+ profiles: [seeder]
+ restart: unless-stopped
+ command: ["pnpm", "--silent", "archilyzer", "seed"]
+ network_mode: service:seeder-vpn
+ depends_on:
+ seeder-vpn:
+ condition: service_healthy
+ env_file:
+ - path: .env
+ required: false
+ environment:
+ TRANSCRIPTS_DIR: /data/transcripts
+ SETTINGS_FILE: /data/config/settings.json
+ volumes:
+ # Read-only: the seeder reads site.json, the playable manifests and the
+ # copies; it writes nothing to the corpus. A playable root on another
+ # drive (beside a saved-video store moved off the corpus volume) needs
+ # its own read-only bind mount at the same absolute path, in a
+ # docker-compose.override.yml.
+ - corpus:/data/transcripts:ro
+ - config:/data/config:ro
+
+ # The pilot's self-hosted tracker (WebSocket + HTTP), in the same namespace,
+ # so its announces and its address are the tunnel's too. TLS (wss://) is
+ # whatever terminates in front of the forwarded port.
+ tracker:
+ image: archilyzer:${ARCHILYZER_TAG:-local}
+ profiles: [seeder]
+ restart: unless-stopped
+ command: ["pnpm", "--silent", "archilyzer", "tracker", "--host", "0.0.0.0", "--port", "${SEEDER_TRACKER_PORT:-8000}"]
+ network_mode: service:seeder-vpn
+ depends_on:
+ seeder-vpn:
+ condition: service_healthy
+ env_file:
+ - path: .env
+ required: false
+ environment:
+ TRANSCRIPTS_DIR: /data/transcripts
+ SETTINGS_FILE: /data/config/settings.json
+ volumes:
+ # It reads the playable manifests to know which infohashes it tracks.
+ - corpus:/data/transcripts:ro
+ - config:/data/config:ro
diff --git a/editor/CHANGELOG.md b/editor/CHANGELOG.md
@@ -1,6 +1,9 @@
# Changelog
## [Unreleased]
+- **A home seeder of last resort, behind a VPN.** `archilyzer seed` seeds the playable torrents of the sites named in the new `settings.seeder` (`sites`, `trackers`, `maxUploadKiBps`, `maxConnections`, `pollSeconds`, `standbyAfterSeconds`, `bindInterface`; SETTINGS.md) to desktop clients over TCP and to browsers over WebRTC — but each torrent only while no other seeder has it: other seeders seen on every poll for `standbyAfterSeconds` puts that torrent on standby (it stops announcing and closes its peers, keeping the data), and it comes back at once when a leecher is waiting with no other source, or after the same window with no other seeder. Every change is logged with its reason. No DHT, no local discovery, no UPnP. `archilyzer tracker` is a self-hosted HTTP + WebSocket tracker that tracks only those torrents. `docker-compose.seeder.yml` (profile `seeder`) runs both inside a WireGuard container's network namespace (gluetun, its firewall always on), so a tunnel that is down means no network, never the home connection; the WireGuard config is yours (`SEEDER_WG_CONF`, required, mounted read-only). `archilyzer doctor` compares the seeder's egress address with the host's and fails when they are the same; it says "seeder not configured" until `seeder.sites` names a site.
+- **Saved videos can be made browser-playable, with a torrent each.** `pnpm ops prepare-playable` (`POST /api/ops/prepare-playable`) and `archilyzer media playable <slug>` remux each of a channel's saved containers — without re-encoding (`-c copy`) — into an mp4 with its index in front, or a webm when it already is one (VP9/AV1 with Opus), drop subtitles, metadata and chapters, and make one single-file torrent of the copy: named `<id>.<ext>`, no web seed, no comment, no "created by", 256 KiB–1 MiB pieces. They go to `playable/<slug>/<id>/` beside the saved-video store, listed in `playable/<slug>/playable.json` with each infohash. `"trackers"` is the announce list written into each torrent (none by default); it is not part of the infohash, so the same torrent can be announced elsewhere later. A video already prepared from the same source (by sha256) is skipped, so a re-run is a no-op; a codec a browser cannot play without re-encoding (HEVC, MPEG-4 Part 2) is listed and left alone.
+- **A channel's videos can get their media from a local archive.** `pnpm ops attach-media` (`POST /api/ops/attach-media`) and `archilyzer media attach <slug> <source>` take `{"slug", "source"}` — an absolute path to a directory, a `.zip` (read in place: a stored entry is copied straight out of it, with no temp dir) or a `.7z` — and put each held video's file into the saved-video store as its source container, so clip windows and report clips can be cut from it with nothing fetched. The id is the folder's trailing `(<id>)`, else the file's yt-dlp suffix; `"items": [{"id", "path"}]` names exact files and `"match"` narrows the folders. The pointer records where the file came from: `origin: {kind: "local-archive", archive, entry, sha256, attachedAt}`. A video that already has a saved container is left alone unless `"replace": true`; `"createRecords": true` writes a record for a video the channel does not hold (from the folder's yt-dlp `.info.json`, else its `description.txt` and name); `[LOST]` folders are listed and never attached. `"dryRun": true` lists what each folder is, and the held videos the archive has no media for, and writes nothing. The archive is never written to.
- **Reports can be checked, their quotes verified and their video attached from the command line.** `archilyzer reports check <site> [--reports a,b] [--allow-missing-media]` says what the site's compose would say about its reports — each report validated, every cited quote checked against its record, every cited moment's prepared media present and current, the report's video under the publish limit — without a build, and exits 1 with the list; `--reports` checks those reports, drafts not yet in site.json included. `archilyzer reports verify-quotes <report.json> [--json]` runs compose's own quote check over every video, audio and post quote of one report and prints each one's score and the transcript it matched best, and, where the record has an `en-orig` track, that track's score: a quote that matches a served `en` track but not `en-orig` — the words as spoken — is reported, though compose would pass it. `archilyzer reports attach-video <report.json> <video> [--poster <image>] [--caption <line>]` makes the video the report's: an H.264 mp4 under the 24 MiB limit is remuxed, anything else encoded to fit (refused, with its length, when it would be unwatchable), written beside report.json as `video.mp4` with a poster — the one given, the one it had, or a frame of the video — and `video` set in report.json.
- **A file transcription is timed from the engine, and a short one does not queue behind batches.** `pnpm ops transcribe`'s `durationMs` is now the engine's own time — from the moment a worker took the job — and the new `waitedMs` is how long it waited for a free worker; the job's log says both. A file of up to 15 minutes of audio (a window, usually) waits in the worker pool ahead of every parked transcription, a manual batch's next video included, not only ahead of the transcription lane; a longer file keeps its place behind batches queued before it. Nothing running is interrupted. Needs a restart of the editor.
- **Heavy work takes turns, above a memory floor.** A publish stage's `next build` — a site's, the hub's, the homepage's — now waits for the machine's one heavy slot, which every e2e run and any `pnpm heavy -- <cmd>` (a video render) take too, and then until at least 6000 MB is available; the stage's log says whom it waits behind ("waiting for the heavy slot — held by …") or how much memory there is ("waiting for memory — 4210 MB available, the floor is 6000 MB"). Cancel still stops it. `HEAVY_MIN_FREE_MB` moves the floor (`0` turns it off) and `HEAVY=0` skips the gate. Needs a restart of the editor.
diff --git a/editor/app/api/ops/attach-media/route.test.ts b/editor/app/api/ops/attach-media/route.test.ts
@@ -0,0 +1,91 @@
+import test from "node:test";
+import assert from "node:assert/strict";
+import { mkdir, mkdtemp, readFile, readdir, rm, writeFile } from "node:fs/promises";
+import os from "node:os";
+import path from "node:path";
+import { makeZip } from "../../../../../common/testing/makeZip";
+
+// Run with:
+// pnpm -C editor exec tsx --test "app/api/ops/attach-media/route.test.ts"
+//
+// The body's shape, the refusals that come before any job, and one dry run
+// over a synthetic zip, answered from a temp corpus. Nothing is attached.
+
+const ROOT = await mkdtemp(path.join(os.tmpdir(), "attach-media-route-"));
+// Set before the route (and getPaths, which caches) is first imported.
+process.env.WORKER_TOKEN = "test-token";
+process.env.TRANSCRIPTS_DIR = ROOT;
+process.env.SETTINGS_FILE = path.join(ROOT, "settings.json");
+const { POST } = await import("./route");
+const { getRegistry } = await import("yt-dlp-transcript-common/jobs/registry");
+const { getPaths } = await import("yt-dlp-transcript-common/lib/paths");
+test.after(() => rm(ROOT, { recursive: true, force: true }));
+
+const CH = path.join(ROOT, "channels");
+await mkdir(path.join(CH, "tism", "data", "aaaaaaaaaaa"), { recursive: true });
+await writeFile(path.join(CH, "tism", "config.json"), JSON.stringify({ handling: "youtube", name: "tism", platform: "youtube" }));
+await writeFile(path.join(CH, "tism", "data", "aaaaaaaaaaa", "metadata.info.json"), JSON.stringify({ id: "aaaaaaaaaaa" }));
+const ZIP = path.join(ROOT, "archive.zip");
+await writeFile(
+ ZIP,
+ makeZip([
+ { name: "YouTube/001 - A - (aaaaaaaaaaa)/A-aaaaaaaaaaa.mp4", data: Buffer.alloc(1000, 1) },
+ { name: "YouTube/002 [LOST] - B - (bbbbbbbbbbb)/source.txt", data: "x" },
+ { name: "YouTube/003 - C - (ccccccccccc)/ccccccccccc.mp4", data: Buffer.alloc(500, 2) },
+ ]),
+);
+
+async function post(
+ body: Record<string, unknown>,
+): Promise<{ status: number; json: { ok?: boolean; error?: string; jobId?: string } }> {
+ const res = await POST(
+ new Request("http://localhost/api/ops/attach-media", {
+ method: "POST",
+ headers: { authorization: "Bearer test-token", "content-type": "application/json" },
+ body: JSON.stringify(body),
+ }),
+ );
+ return { status: res.status, json: (await res.json()) as { ok?: boolean; error?: string; jobId?: string } };
+}
+
+test("refusals before any job: keys, slug, source, items, match, channel", async () => {
+ const cases: [Record<string, unknown>, RegExp][] = [
+ [{ slug: "tism" }, /"source" is required/],
+ [{ source: ZIP }, /"slug" is required/],
+ [{ slug: "../x", source: ZIP }, /not a valid channel slug/],
+ [{ slug: "tism", source: ZIP, force: true }, /unknown key\(s\): force/],
+ [{ slug: "tism", source: "relative/archive.zip" }, /must be an absolute path/],
+ [{ slug: "tism", source: ZIP, items: [] }, /non-empty array/],
+ [{ slug: "tism", source: ZIP, items: [{ id: "..", path: "x" }] }, /not a video id/],
+ [{ slug: "tism", source: ZIP, items: [{ id: "a", path: "x", size: 1 }] }, /unknown key\(s\): size/],
+ [{ slug: "tism", source: ZIP, items: [{ id: "a", path: "x" }], match: "x" }, /"items" or "match", not both/],
+ [{ slug: "tism", source: ZIP, match: "(" }, /not a valid regex/],
+ [{ slug: "tism", source: ZIP, dryRun: "yes" }, /"dryRun" must be a boolean/],
+ [{ slug: "no-such", source: ZIP }, /Channel "no-such" not found/],
+ ];
+ for (const [body, re] of cases) {
+ const r = await post(body);
+ assert.equal(r.status, 400, JSON.stringify(body));
+ assert.match(r.json.error!, re, JSON.stringify(body));
+ assert.equal(r.json.jobId, undefined);
+ }
+});
+
+test("a dry run is a job whose log classifies the archive and writes nothing", async () => {
+ const r = await post({ slug: "tism", source: ZIP, dryRun: true });
+ assert.equal(r.status, 200, JSON.stringify(r.json));
+ assert.ok(r.json.jobId);
+ for (let i = 0; i < 100; i++) {
+ const status = getRegistry().get(r.json.jobId!)?.status;
+ if (status === "done" || status === "failed") break;
+ await new Promise((res) => setTimeout(res, 50));
+ }
+ assert.equal(getRegistry().get(r.json.jobId!)?.status, "done");
+ const log = await readFile(path.join(getPaths().jobsDir, `${r.json.jobId}.log`), "utf8");
+ const summary = JSON.parse(log.split("\n").find((l) => l.startsWith("summary: "))!.slice("summary: ".length));
+ assert.equal(summary.dryRun, true);
+ assert.equal(summary.counts.attach, 1);
+ assert.deepEqual(summary.notHeld, ["ccccccccccc"]);
+ assert.equal(summary.lost, 1);
+ assert.deepEqual(await readdir(path.join(CH, "tism", "data", "aaaaaaaaaaa")), ["metadata.info.json"]);
+});
diff --git a/editor/app/api/ops/attach-media/route.ts b/editor/app/api/ops/attach-media/route.ts
@@ -0,0 +1,70 @@
+import { attachMediaAction } from "../../../channels/[slug]/attachMediaActions";
+import {
+ OpsInputError,
+ jobResponse,
+ ops,
+ optBool,
+ optString,
+ reqSlug,
+ reqString,
+ reqVideoId,
+ type OpsBody,
+} from "../_lib";
+
+export const dynamic = "force-dynamic";
+
+// POST { slug, source, items?: [{id, path}], match?, createRecords?, replace?, dryRun? }
+// -> { ok: true, jobId }
+//
+// Local media attached to a channel's held videos (release 21 D1): each
+// video's file copied out of `source` — an absolute path to a directory, a
+// .zip (read in place) or a .7z — into the saved-video store, as one
+// `attach-media` job on the channel's queue (controller/attachMedia.ts). The
+// id comes from the folder's trailing "(<id>)", else the file's yt-dlp
+// suffix; `items` names exact files instead (each `path` is the entry's path
+// inside the source), `match` narrows the folders by a case-insensitive regex.
+// `createRecords` writes a record for a video the channel does not hold;
+// `replace` re-attaches over an existing saved container. `dryRun` logs what
+// each folder is (attach, not held, already attached, [LOST], no media,
+// unmatched, ambiguous) and the held videos the archive has no media for, and
+// writes nothing. The job's log ends with a `summary: {…}` line.
+function optItems(body: OpsBody): { id: string; path: string }[] | undefined {
+ const v = body.items;
+ if (v === undefined) return undefined;
+ if (!Array.isArray(v) || v.length === 0) {
+ throw new OpsInputError('"items" must be a non-empty array of { "id", "path" }');
+ }
+ return v.map((entry, i) => {
+ if (typeof entry !== "object" || entry === null || Array.isArray(entry)) {
+ throw new OpsInputError(`"items[${i}]" must be an object { "id", "path" }`);
+ }
+ const extra = Object.keys(entry).filter((k) => k !== "id" && k !== "path");
+ if (extra.length) {
+ throw new OpsInputError(
+ `"items[${i}]" has unknown key(s): ${extra.join(", ")} — an item is { "id", "path" }`,
+ );
+ }
+ return {
+ id: reqVideoId(entry as OpsBody, "id"),
+ path: reqString(entry as OpsBody, "path"),
+ };
+ });
+}
+
+export async function POST(request: Request) {
+ return ops(
+ request,
+ ["slug", "source", "items", "match", "createRecords", "replace", "dryRun"],
+ async (body) =>
+ jobResponse(
+ await attachMediaAction(reqSlug(body, "slug"), {
+ source: reqString(body, "source"),
+ items: optItems(body),
+ match: optString(body, "match"),
+ createRecords: optBool(body, "createRecords"),
+ replace: optBool(body, "replace"),
+ dryRun: optBool(body, "dryRun"),
+ }),
+ ),
+ );
+}
diff --git a/editor/app/api/ops/prepare-playable/route.test.ts b/editor/app/api/ops/prepare-playable/route.test.ts
@@ -0,0 +1,66 @@
+import test from "node:test";
+import assert from "node:assert/strict";
+import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
+import os from "node:os";
+import path from "node:path";
+
+// Run with:
+// pnpm -C editor exec tsx --test "app/api/ops/prepare-playable/route.test.ts"
+//
+// The body's shape and the refusals that come before any job, then one dry
+// run over a channel with no saved containers. Nothing is remuxed.
+
+const ROOT = await mkdtemp(path.join(os.tmpdir(), "prepare-playable-route-"));
+process.env.WORKER_TOKEN = "test-token";
+process.env.TRANSCRIPTS_DIR = ROOT;
+process.env.SETTINGS_FILE = path.join(ROOT, "settings.json");
+const { POST } = await import("./route");
+const { getRegistry } = await import("yt-dlp-transcript-common/jobs/registry");
+const { getPaths } = await import("yt-dlp-transcript-common/lib/paths");
+test.after(() => rm(ROOT, { recursive: true, force: true }));
+
+await mkdir(path.join(ROOT, "channels", "tism", "data", "aaaaaaaaaaa"), { recursive: true });
+await writeFile(path.join(ROOT, "channels", "tism", "config.json"), JSON.stringify({ handling: "youtube", name: "tism" }));
+
+async function post(body: Record<string, unknown>) {
+ const res = await POST(
+ new Request("http://localhost/api/ops/prepare-playable", {
+ method: "POST",
+ headers: { authorization: "Bearer test-token", "content-type": "application/json" },
+ body: JSON.stringify(body),
+ }),
+ );
+ return { status: res.status, json: (await res.json()) as { ok?: boolean; error?: string; jobId?: string } };
+}
+
+test("refusals before any job", async () => {
+ const cases: [Record<string, unknown>, RegExp][] = [
+ [{}, /"slug" is required/],
+ [{ slug: "tism", force: 1 }, /unknown key\(s\): force/],
+ [{ slug: "tism", ids: [] }, /non-empty array/],
+ [{ slug: "tism", ids: ["../x"] }, /not a video id/],
+ [{ slug: "tism", trackers: "wss://x" }, /must be an array/],
+ [{ slug: "tism", trackers: ["file:///etc/passwd"] }, /not a tracker URL/],
+ [{ slug: "tism", root: "relative" }, /must be an absolute path/],
+ [{ slug: "nope" }, /Channel "nope" not found/],
+ ];
+ for (const [body, re] of cases) {
+ const r = await post(body);
+ assert.equal(r.status, 400, JSON.stringify(body));
+ assert.match(r.json.error!, re, JSON.stringify(body));
+ }
+});
+
+test("a dry run over a channel with no saved containers is a job that says so", async () => {
+ const r = await post({ slug: "tism", dryRun: true, root: path.join(ROOT, "playable"), trackers: ["wss://tracker.example/announce"] });
+ assert.equal(r.status, 200, JSON.stringify(r.json));
+ for (let i = 0; i < 100; i++) {
+ const s = getRegistry().get(r.json.jobId!)?.status;
+ if (s === "done" || s === "failed") break;
+ await new Promise((res) => setTimeout(res, 50));
+ }
+ assert.equal(getRegistry().get(r.json.jobId!)?.status, "done");
+ const log = await readFile(path.join(getPaths().jobsDir, `${r.json.jobId}.log`), "utf8");
+ assert.match(log, /0 video\(s\) with a saved container/);
+ assert.match(log, /summary: \{"dryRun":true/);
+});
diff --git a/editor/app/api/ops/prepare-playable/route.ts b/editor/app/api/ops/prepare-playable/route.ts
@@ -0,0 +1,41 @@
+import { preparePlayableAction } from "../../../channels/[slug]/playableActions";
+import { OpsInputError, jobResponse, ops, optBool, optString, reqSlug, reqStringArray, reqVideoId, type OpsBody } from "../_lib";
+
+export const dynamic = "force-dynamic";
+
+// POST { slug, ids?: string[], trackers?: string[], root?, dryRun? }
+// -> { ok: true, jobId }
+//
+// Playable copies and their torrents (release 21 D3): each of the channel's
+// saved containers remuxed losslessly (`-c copy`) into mp4 (+faststart) or
+// webm, and one single-file torrent per copy, under the playable root —
+// `playable/` beside the saved-video store, unless `root` names another — as
+// one `prepare-playable` job (controller/preparePlayable.ts). `ids` narrows
+// it to those videos; `trackers` is the announce list written into each
+// .torrent (none by default; it is outside the info dict, so the infohash
+// does not depend on it). A video already prepared from the same source
+// (by sha256) is skipped. `dryRun` logs each decision and writes nothing.
+function optIds(body: OpsBody): string[] | undefined {
+ if (body.ids === undefined) return undefined;
+ return reqStringArray(body, "ids").map((id) => reqVideoId({ id }, "id"));
+}
+
+function optTrackers(body: OpsBody): string[] | undefined {
+ if (body.trackers === undefined) return undefined;
+ if (!Array.isArray(body.trackers)) throw new OpsInputError('"trackers" must be an array of announce URLs');
+ if (body.trackers.length === 0) return [];
+ return reqStringArray(body, "trackers");
+}
+
+export async function POST(request: Request) {
+ return ops(request, ["slug", "ids", "trackers", "root", "dryRun"], async (body) =>
+ jobResponse(
+ await preparePlayableAction(reqSlug(body, "slug"), {
+ ids: optIds(body),
+ trackers: optTrackers(body),
+ root: optString(body, "root"),
+ dryRun: optBool(body, "dryRun"),
+ }),
+ ),
+ );
+}
diff --git a/editor/app/channels/[slug]/attachMediaActions.ts b/editor/app/channels/[slug]/attachMediaActions.ts
@@ -0,0 +1,94 @@
+"use server";
+
+import path from "node:path";
+import { safeRevalidate } from "../../lib/safeRevalidate";
+import { getPaths } from "yt-dlp-transcript-common/lib/paths";
+import { channelQueueKey } from "yt-dlp-transcript-common/lib/queueKeys";
+import { readChannelConfig } from "yt-dlp-transcript-common/controller/channels";
+import {
+ attachMedia,
+ summaryOf,
+} from "yt-dlp-transcript-common/controller/attachMedia";
+import {
+ runManagedFunction,
+ type StreamActionResult,
+} from "yt-dlp-transcript-common/jobs/streamCommand";
+
+// LOCAL MEDIA ATTACHED TO A CHANNEL'S HELD VIDEOS (release 21 D1): the
+// `attach-media` job, as `pnpm ops attach-media` posts it
+// (`/api/ops/attach-media`). The work and its rules are
+// common/controller/attachMedia.ts; this is the enqueue and the refusals a
+// caller gets before any job exists.
+//
+// On the channel's own queue: nothing is fetched, so no platform queue is owed
+// a turn, and a second attach on the same channel waits behind the first
+// rather than racing it for the same pointers. The kind's `needsMedia` is what
+// keeps it off a channel whose media is moving or unreachable.
+export async function attachMediaAction(
+ slug: string,
+ opts: {
+ source: string;
+ items?: { id: string; path: string }[];
+ match?: string;
+ createRecords?: boolean;
+ replace?: boolean;
+ dryRun?: boolean;
+ },
+): Promise<StreamActionResult> {
+ const paths = getPaths();
+ const channelConfig = await readChannelConfig(paths, slug);
+ if (!channelConfig) {
+ return { ok: false, error: `Channel "${slug}" not found` };
+ }
+ const source = opts.source.trim();
+ if (!path.isAbsolute(source)) {
+ return { ok: false, error: `"source" must be an absolute path (a directory, a .zip or a .7z)` };
+ }
+ if (opts.items !== undefined && opts.match !== undefined) {
+ return { ok: false, error: 'Send "items" or "match", not both: items name the files exactly' };
+ }
+ if (opts.match !== undefined) {
+ try {
+ new RegExp(opts.match, "i");
+ } catch (e) {
+ return { ok: false, error: `"match" is not a valid regex: ${(e as Error).message}` };
+ }
+ }
+ return runManagedFunction({
+ kind: "attach-media",
+ queueKey: channelQueueKey(slug),
+ paths,
+ channelSlug: slug,
+ fn: async (onLog, signal, _setProgress, ctx) => {
+ const result = await attachMedia({
+ paths,
+ slug,
+ channelConfig,
+ source,
+ ...(opts.items ? { items: opts.items } : {}),
+ ...(opts.match !== undefined ? { match: opts.match } : {}),
+ ...(opts.createRecords ? { createRecords: true } : {}),
+ ...(opts.replace ? { replace: true } : {}),
+ ...(opts.dryRun ? { dryRun: true } : {}),
+ onLog,
+ signal,
+ drainSignal: ctx.drainSignal,
+ });
+ if (!opts.dryRun) {
+ safeRevalidate([
+ `/channels/${slug}`,
+ "/channels",
+ ...result.attached.map((id) => `/channels/${slug}/videos/${id}`),
+ ]);
+ }
+ if (result.failed.length > 0 && result.attached.length === 0) {
+ throw new Error(
+ `${result.failed.length} failed, none attached: ${JSON.stringify(summaryOf(result).failed)}`,
+ );
+ }
+ if (result.stopped && result.stopped !== "drained" && result.stopped !== "cancelled") {
+ throw new Error(`stopped: ${result.stopped}`);
+ }
+ },
+ });
+}
diff --git a/editor/app/channels/[slug]/playableActions.ts b/editor/app/channels/[slug]/playableActions.ts
@@ -0,0 +1,67 @@
+"use server";
+
+import path from "node:path";
+import { safeRevalidate } from "../../lib/safeRevalidate";
+import { getPaths } from "yt-dlp-transcript-common/lib/paths";
+import { readChannelConfig } from "yt-dlp-transcript-common/controller/channels";
+import { preparePlayable } from "yt-dlp-transcript-common/controller/preparePlayable";
+import {
+ runManagedFunction,
+ type StreamActionResult,
+} from "yt-dlp-transcript-common/jobs/streamCommand";
+
+// PLAYABLE COPIES AND THEIR TORRENTS (release 21 D3): the `prepare-playable`
+// job, as `pnpm ops prepare-playable` posts it (`/api/ops/prepare-playable`).
+// The work and its rules are common/controller/preparePlayable.ts.
+//
+// ONE QUEUE MACHINE-WIDE: a remux reads and writes gigabytes on the archive
+// drive, and two channels' worth at once only thrash it.
+const PLAYABLE_QUEUE = "playable";
+
+export async function preparePlayableAction(
+ slug: string,
+ opts: { ids?: string[]; trackers?: string[]; root?: string; dryRun?: boolean },
+): Promise<StreamActionResult> {
+ const paths = getPaths();
+ const channelConfig = await readChannelConfig(paths, slug);
+ if (!channelConfig) {
+ return { ok: false, error: `Channel "${slug}" not found` };
+ }
+ if (opts.root !== undefined && !path.isAbsolute(opts.root)) {
+ return { ok: false, error: `"root" must be an absolute path` };
+ }
+ for (const tr of opts.trackers ?? []) {
+ if (!/^(wss?|https?|udp):\/\/[^\s]+$/i.test(tr)) {
+ return { ok: false, error: `"${tr}" is not a tracker URL (wss://, ws://, https://, http:// or udp://)` };
+ }
+ }
+ return runManagedFunction({
+ kind: "prepare-playable",
+ queueKey: PLAYABLE_QUEUE,
+ paths,
+ channelSlug: slug,
+ fn: async (onLog, signal, _setProgress, ctx) => {
+ const result = await preparePlayable({
+ paths,
+ slug,
+ channelConfig,
+ ...(opts.root ? { root: opts.root } : {}),
+ ...(opts.ids ? { ids: opts.ids } : {}),
+ ...(opts.trackers ? { trackers: opts.trackers } : {}),
+ ...(opts.dryRun ? { dryRun: true } : {}),
+ onLog,
+ signal,
+ drainSignal: ctx.drainSignal,
+ });
+ if (!opts.dryRun && result.prepared.length > 0) {
+ safeRevalidate([`/channels/${slug}`]);
+ }
+ if (result.failed.length > 0 && result.prepared.length === 0) {
+ throw new Error(`${result.failed.length} failed, none prepared`);
+ }
+ if (result.stopped && result.stopped !== "drained" && result.stopped !== "cancelled") {
+ throw new Error(`stopped: ${result.stopped}`);
+ }
+ },
+ });
+}
diff --git a/plans/release-21.md b/plans/release-21.md
@@ -152,6 +152,148 @@ D1 ──► D3 (copies + torrents) ──► D4 (publish torrents) ──► D5
D1, D3 and D4b touch no file owned by tracks A/B/C and may start now. D2 waits for A5. D4 and D5 wait until release
19 merges (the ops routes, `_cli.ts`, the `site.json` schema docs).
+## Record
+
+### Track D
+
+One Opus implementer, branch `worktree-agent-a4b4aebbb6c0d28bb` off `5bc20d32` (`r19/integration` with this plan),
+`r19/integration` merged in before this record (`2e3a9395`: tracks B and C — the docs cli, the heavy slot). Slices
+D1, D3, D4b, in that order; D2, D4, D5 are not this track's. Nothing ran against the real corpus or the real
+archive beyond read-only listings of the archive's central directory (`unzip -l`, `zipinfo`, and the new zip
+reader over it); every fixture is synthetic.
+
+#### D1, as shipped — attach local media to held videos
+
+**What was found before building.**
+- The archive is ZIP64 (790 entries, 36.45 GB, every entry stored): its later offsets do not fit 32 bits. No zip
+ library is in the workspace, so `common/lib/zipReader.ts` reads the central directory itself (EOCD, the ZIP64
+ locator and record, the 0x0001 extra, the Info-ZIP Unicode-path extra) and copies a stored entry as a byte range
+ of the archive, a deflated one through `inflateRaw`, checking the size and CRC32 against the central directory.
+ Over the real archive (read-only) it lists the 790 entries and their byte total matches `zipinfo`'s exactly.
+- One media file name in the archive is non-ASCII (a full-width `｜` and `？`); the folder's `(<id>)` names it.
+- The archive's folder classes, from the listing alone (no corpus read): 182 folders with files, 169 with media,
+ 10 `[LOST]`, 2 with an id and no media, and the archive's root (no media, no id).
+- `persistSourceVideo` MOVES `<videoDir>/<sourceFilename>`. Copying a 36 GB archive into the corpus SSD first would
+ be the wrong disk twice, so it gains `sourcePath` (the file to move, a staging file beside the store dir on the
+ store's disk — a same-filesystem rename) and `sha256` (recorded when the caller hashed while copying).
+- Every consumer of `SavedVideoOrigin` reads `requestedBy` as required (the video page's "Requested by", the clip
+ requester line in `videoActions.ts`, which this track may not touch). So the local-archive origin keeps
+ `requestedBy: "attach-media"` and adds `kind: "local-archive"`, `archive`, `entry`, `sha256`, `attachedAt`;
+ `parseOrigin` reads them only under that kind, and a pointer without `kind` parses exactly as before.
+- The non-yt-dlp writer of `metadata.info.json` for a new record is `withMetadataHistory` around
+ `writeJsonAtomic(…, {indent: 0, newline: false})`, as `archiveOrgDownload` writes its records; a new writer
+ `local-archive` joins `METADATA_HISTORY_WRITERS`.
+
+**What it does.** `common/lib/attachMediaPlan.ts` (pure): folders grouped by parent; the id from the folder's
+trailing `(<11-char id>)`, else the file's `-<id>.`, ` [<id>].` or bare `<id>.`; classes `attach`, `create`,
+`already-attached`, `not-held`, `lost`, `no-media`, `unmatched`, `ambiguous` (two video files in a folder, or two
+folders claiming one id), plus the held videos no folder supplied (only when the whole archive was considered).
+`common/controller/attachMedia.ts`: sources are a directory, a `.zip` read in place, or a `.7z` (listed with
+`7z l -slt`, one entry extracted with `7z e -so -spd` into the staging file); the store's root must exist (an
+unmounted drive is never materialised), `assertSavedVideosStoreWritable` before each copy, size plus 2 GiB free
+on the store's disk or the run stops (a re-run resumes); drain and cancel between videos, a cancel aborts the copy
+and removes its staging file; `keepReason: "pin"`; refuses over a pointer unless `replace` (a replaced container of
+another name is removed after the new pointer is written). `createRecords` writes the folder's own yt-dlp
+`.info.json` (refused when its `id` is another video's) or one built from `description.txt` ("Published on …" →
+`upload_date`, last key) and the folder name, the duration from ffprobe, and appends `youtube <id>` to the
+channel's `archive`. The log ends with `summary: {…}`. Job kind `attach-media` (drainable, `needsMedia`, an ingest
+kind — its pointers and records are what the index reads; not replayable; the channel's queue). The action is
+`editor/app/channels/[slug]/attachMediaActions.ts` (a new file); `POST /api/ops/attach-media` on the `_lib.ts`
+precedent; one `ACTIONS` entry and one usage paragraph in `scripts/archilyzer-ops.mjs`; `archilyzer media attach
+<slug> <source>` (offline; asks the channel's media guard itself).
+
+#### D3, as shipped — playable copies and torrents
+
+`common/lib/playableMedia.ts` (pure) is the decision table: h264/vp9/av1 with aac/opus/mp3 (or no audio) → mp4
+`+faststart`; vp9/vp8/av1 with opus/vorbis in a `.webm` → webm with `-cues_to_front 1`; anything else (HEVC,
+MPEG-4 Part 2, AC-3 audio, no video) is listed as not playable, never re-encoded. Only the first real video stream
+(`0:V:0`, not cover art) and the first audio are kept; subtitles, data, global metadata and chapters are dropped;
+`+bitexact`, so the same source remuxes to the same bytes (tested). The measured mix in this plan is every row of
+the table's playable half. `common/controller/preparePlayable.ts`: copies, torrents and `playable.json` under
+`playable/<slug>/<id>/` beside the saved-video store's real directory (or `root`); keyed by the source's sha256 (a
+pointer with none is hashed once and the hash recorded on it), so a re-run with the files on disk at their size is
+a no-op; each copy written to a `.part` and renamed; the manifest rewritten after each video. Each remux takes the
+heavy slot (`scripts/queue-lock.mjs --heavy`, release 19 B1) per video, so a long run interleaves with builds and
+e2e; a checkout without the gate runs ffmpeg directly. **The torrent:** `create-torrent` with `name` `<id>.<ext>`,
+piece length a power of two in 256 KiB–1 MiB scaled to ~2,048 pieces, a fixed creation date, the announce list from
+the `trackers` parameter (an explicit empty list, never create-torrent's default public trackers), and NOT passed:
+`createdBy`, `comment`, `urlList`, `private` — even `private: false` would put `private: 0` inside the info dict and
+move the infohash. Tested: same bytes and piece length → same infohash whatever the path or the trackers; another
+piece length → another; the .torrent byte-identical across runs. Job kind `prepare-playable` (drainable,
+`needsMedia`, ingest — D4's build reads the manifest; one `playable` queue machine-wide), `POST
+/api/ops/prepare-playable`, one ops row, `archilyzer media playable <slug>`.
+
+#### D4b, as shipped — the seeder of last resort, behind a VPN
+
+- **The state machine** (`common/lib/lastResortSeeder.ts`, pure): a torrent starts seeding; other seeders (the
+ scrape's `complete` minus itself, or connected seeds, whichever is more) on every poll for `standbyAfterSeconds`
+ → standby; in standby, no other seeder and a leecher waiting (the scrape's `incomplete` or a wire) → seeding at
+ once, or no other seeder on every poll for the resume window → seeding; trackers silent for the window while in
+ standby → seeding (a last resort assumes it is the only one). A transition restarts both clocks. The resume window
+ is `standbyAfterSeconds` (no separate key). 13 tests against a scripted scrape: both edges, hysteresis on each,
+ a flapping swarm that never flaps it, the plan's rollout step 7 as a script.
+- **The service** (`common/controller/seeder.ts`): WebTorrent 3 in node — TCP, and WebRTC through
+ `webrtc-polyfill` → `node-datachannel` (WebTorrent's own dependency chain). Standby removes the torrent from the
+ client with its data kept (`stopped` announced, peers closed); seeding adds it back with `skipVerify` after a
+ size check of the copy. No DHT, LSD, UPnP/NAT-PMP or web seeds. Scrapes are one request per tracker for every
+ infohash, merged per torrent by max; wires are counted by peer (two peers that dialled each other hold two
+ wires — found by the integration test, which first saw "2 other seeders"). `bindInterface` held: the seeder
+ waits for it and drops every torrent when it disappears. Targets are re-read each poll from the playable
+ manifests of every channel of `settings.seeder.sites`.
+- **The tracker** (`archilyzer tracker`): bittorrent-tracker, HTTP + WebSocket, no UDP, no stats page, bound to
+ `--host` (default 127.0.0.1), refusing every infohash not in the manifests (re-read at most once a minute).
+- **Settings** `seeder: {sites, trackers, maxUploadKiBps, maxConnections, pollSeconds, standbyAfterSeconds,
+ bindInterface}` (`common/lib/seederSettings.ts`, SETTINGS.md and settings.json.example regenerated); not
+ configured = `sites: []`, and `archilyzer seed` refuses to start.
+- **The VPN** (`docker-compose.seeder.yml`, profile `seeder`, an overlay on `docker-compose.yml`): gluetun
+ `v3.41.3` (custom provider, WireGuard; its firewall is always on in v3 — there is no switch, and the file names
+ none), the seeder and the tracker with `network_mode: service:seeder-vpn`, no networks or ports of their own,
+ waiting for a healthy tunnel, the corpus read-only. The WireGuard config is the operator's: `${SEEDER_WG_CONF:?…}`
+ mounted read-only — compose refuses the file without it (checked with `docker compose … config`). Inbound only
+ through `SEEDER_VPN_INPUT_PORTS` (empty by default). `lib/seederCompose.test.ts` pins that construction.
+- **Doctor** `seeder` section: "seeder not configured" (a note, no network) while `sites` is empty; else the host's
+ egress address and the seeder container's (`docker exec archilyzer-seeder-1 node -e fetch(…)`, one echo URL):
+ the same address FAILS, either unanswered warns, different is ok.
+- **Measured (localhost, the integration tests):** a TCP leecher completes from the seeder; it then seeds, the
+ seeder logs `seeding → standby — 1 other seeder(s) on every poll for 1s`; the leecher leaves and it logs
+ `standby → seeding — no other seeder on every poll for 1s`. A WebRTC leecher completes through the WebSocket
+ tracker over a `webrtc` wire (node-datachannel). Two node peers that announce in the same instant each answer the
+ other's offer and each keep the connection the other dropped; the test starts the leecher after the seeder's
+ announce is in, as a browser arriving at a running seeder does.
+- **Not measured, owed to the rollout:** whether the VPN's NAT lets browser peers connect (the plan's risk), and
+ the manual network test. WebTorrent's default `rtcConfig` asks public STUN servers for the seeder's address:
+ inside the profile that address is the VPN exit; on a bare host it would be the home address — run it in the
+ profile.
+
+**New dependencies (common):** `create-torrent` 6.1.3, `parse-torrent` 11.0.24 (D3: the torrent and its
+infohash), `webtorrent` 3.0.21 (node ≥ 22; the repo runs 22.23), `bittorrent-tracker` 11.2.3 (D4b). Types for the
+four are `common/types/webtorrent-modules.d.ts` (they ship none). `pnpm-workspace.yaml` `allowBuilds`:
+`node-datachannel: true` (its prebuilt binary); `bufferutil`, `utf-8-validate`, `utp-native`, `ip-set` false (pure-JS
+fallbacks, optional uTP, a package-manager check).
+
+**Commits**
+
+| Commit | What |
+|---|---|
+| `9686fe67` | `common:` attach-media — zip reader, plan, controller, origin kind, `sourcePath`, job kind, ops route + action, ops row, CLI; 33 tests (31 common, 2 editor) |
+| `f1370406` | `common:` prepare-playable — decision table, remux, torrents, manifest, job kind, ops route + action, ops row, CLI; 13 tests (11 common, 2 editor) |
+| `2f704bc6` | `common:` the seeder — state machine, service, tracker, settings, compose profile, doctor; 26 tests |
+| `2e3a9395` | merge `r19/integration` |
+| `0f79dd6e` | `common:` each remux takes the heavy slot (per video; 1 test); COMMANDS.md regenerated |
+| `c066fb37` | `common:` the plan test's titles and ids synthetic throughout |
+
+**Gates.** `pnpm -r --no-bail --workspace-concurrency=1 exec tsc --noEmit` clean at each commit and after the
+merge. Common 3,530 / 3,533 at `2f704bc6` (+68 over the base, 3,465): the three failures are two `autoRunner.test.ts`
+cases (`computeLeafPending …`, `a channel with a move marker …`) that fail identically with the base's
+`jobKinds.ts`, and `fetchPosts.test.ts`'s drain case, which passes alone (load). Editor 163/163 (+4 route tests).
+`test:scripts` 676 + 3 skips at `2f704bc6`, one `queue-lock` banner case failing under load and passing alone (11/11).
+No builds, no e2e (none in this track's budget). After the merge, an operator hold (no heavy runs) left only tsc
+and single test files: preparePlayable 5/5 (the heavy-slot gate included), cli-docs + jobKinds 21/21, and (before the merge) the settings/doctor/CLI/env tests 95/95.
+
+**Left.** `COMMANDS.md` gains the four rows (`media attach`, `media playable`, `seed`, `tracker` — regenerated
+after the merge brought `docs cli`). The rollout's dryRun on the live editor is the parent's. Transcription of
+the three new records is the operator's (`transcribe-one`), as ruled. D2, D4, D5 are other slices.
+
## Rollout (pilot)
1. `pnpm ops attach-media --json '{"slug":"the-incredible-salt-mine","source":"<archive>/Uploads/YouTube.zip","dryRun":true}'`.
diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml
@@ -29,6 +29,9 @@ importers:
'@tanstack/react-virtual':
specifier: ^3.13.0
version: 3.13.24(react-dom@19.2.4(react@19.2.4))(react@19.2.4)
+ bittorrent-tracker:
+ specifier: ^11.2.3
+ version: 11.2.3(supports-color@10.2.2)
class-variance-authority:
specifier: ^0.7.1
version: 0.7.1
@@ -38,6 +41,9 @@ importers:
cmdk:
specifier: ^1.1.1
version: 1.1.1(@types/react-dom@19.2.3(@types/react@19.2.14))(@types/react@19.2.14)(react-dom@19.2.4(react@19.2.4))(react@19.2.4)
+ create-torrent:
+ specifier: ^6.1.3
+ version: 6.1.3
d3-scale:
specifier: ^4.0.2
version: 4.0.2
@@ -68,6 +74,9 @@ importers:
p-limit:
specifier: ^7.3.0
version: 7.3.0
+ parse-torrent:
+ specifier: ^11.0.24
+ version: 11.0.24
radix-ui:
specifier: ^1.6.0
version: 1.6.0(@types/react-dom@19.2.3(@types/react@19.2.14))(@types/react@19.2.14)(react-dom@19.2.4(react@19.2.4))(react@19.2.4)
@@ -92,6 +101,9 @@ importers:
tw-animate-css:
specifier: ^1.4.0
version: 1.4.0
+ webtorrent:
+ specifier: ^3.0.21
+ version: 3.0.21(bare-url@2.5.4)(supports-color@10.2.2)
zod:
specifier: ^4.3.6
version: 4.3.6
@@ -119,7 +131,7 @@ importers:
version: 5.9.3
wrangler:
specifier: 4.147.0
- version: 4.147.0(@types/node@20.19.39)
+ version: 4.147.0(@types/node@20.19.39)(bufferutil@4.1.0)(utf-8-validate@6.0.6)
editor:
dependencies:
@@ -2165,6 +2177,10 @@ packages:
'@sec-ant/readable-stream@0.4.1':
resolution: {integrity: sha512-831qok9r2t8AlxLko40y2ebgSDhenenCatLVeW/uBtnHPyhHOvG0C7TvfgecV+wHzIm5KUICgzmVpWS+IMEAeg==}
+ '@silentbot1/nat-api@0.4.9':
+ resolution: {integrity: sha512-Bm2Fr0sJyGr4B/XgKjQxjGe7Rzs/OlK91OIHsghObxhP3Y4j2y8o7Xjlledu/pxzFEIWaTbZIBSl8ABqoP/WhQ==}
+ engines: {node: '>=10.0.0'}
+
'@sindresorhus/is@7.2.0':
resolution: {integrity: sha512-P1Cz1dWaFfR4IR+U13mqqiGsLFf1KbayybWwdd2vfctdV6hDpUkgCY0nKOLLTMSoRd/jJNjtbqzf13K8DCCXQw==}
engines: {node: '>=18'}
@@ -2320,6 +2336,16 @@ packages:
'@tanstack/virtual-core@3.14.0':
resolution: {integrity: sha512-JLANqGy/D6k4Ujmh8Tr25lGimuOXNiaVyXaCAZS0W+1390sADdGnyUdSWNIfd49gebtIxGMij4IktRVzrdr12Q==}
+ '@thaunknown/simple-peer@10.1.2':
+ resolution: {integrity: sha512-pcG35e815X43Crr/HVmOWlbsx9Jz3KD27HOhy/BwIjQkDlafzJFl0NKb8B+bf+bsnfQL/BB6Z44cMZPQaYHLYQ==}
+
+ '@thaunknown/simple-websocket@9.1.5':
+ resolution: {integrity: sha512-FTrycAOMeJIHxHxH4iiq0EHMlmITvim04FGpwRcrytykg4jCFPaRYJqjo8wvBVx88TYfxrtQCaHhzaKLDKS71w==}
+
+ '@thaunknown/thirty-two@1.0.6':
+ resolution: {integrity: sha512-Uz2kFb0cNeHg7hds8BtI/umcwzaSMUdbkeQgZrIZQbp3gWRFjrvrvBHNEB6SQYChVxC3rWHovKx1Ya/vSRRMYA==}
+ engines: {node: '>=0.2.6'}
+
'@tybys/wasm-util@0.10.1':
resolution: {integrity: sha512-9tTaPJLSiejZKx+Bmog4uSubteqTvFrVrURwkmHixBo0G4seD0zUxp98E1DzUBJxLQ3NPwXrGKDiVjwx/DpPsg==}
@@ -2538,9 +2564,16 @@ packages:
cpu: [x64]
os: [win32]
+ '@webtorrent/http-node@1.3.0':
+ resolution: {integrity: sha512-GWZQKroPES4z91Ijx6zsOsb7+USOxjy66s8AoTWg0HiBBdfnbtf9aeh3Uav0MgYn4BL8Q7tVSUpd0gGpngKGEQ==}
+
'@zeit/schemas@2.36.0':
resolution: {integrity: sha512-7kjMwcChYEzMKjeex9ZFXkt1AyNov9R5HZtjBKVsmVpw7pa7ZtlCGvCBC2vnnXctaYN+aRI61HjIqeetZW5ROg==}
+ abort-controller@3.0.0:
+ resolution: {integrity: sha512-h8lQ8tacZYnR3vNQTgibj+tODHI5/+l06Au2Pcriv/Gmet0eaj4TwWH41sO9wnHDiQsEj19q0drzdWdeAHtweg==}
+ engines: {node: '>=6.5'}
+
acorn-jsx@5.3.2:
resolution: {integrity: sha512-rq9s+JNhf0IChjtDXxllJ7g41oZk5SlXtp0LHwyA5cejwn7vKmKp4pPri6YEePv2PU65sAsegbXtIinmDFDXgQ==}
peerDependencies:
@@ -2551,6 +2584,10 @@ packages:
engines: {node: '>=0.4.0'}
hasBin: true
+ addr-to-ip-port@2.0.0:
+ resolution: {integrity: sha512-9bYbtjamtdLHZSqVIUXhilOryNPiL+x+Q5J/Unpg4VY3ZIkK3fT52UoErj1NdUeVm3J1t2iBEAur4Ywbl/bahw==}
+ engines: {node: '>=12.20.0'}
+
ajv@6.15.0:
resolution: {integrity: sha512-fgFx7Hfoq60ytK2c7DhnF8jIvzYgOMxfugjLOSMHjLIPgenqa7S7oaagATUq99mV6IYvN2tRmC0wnTYX6iPbMw==}
@@ -2644,6 +2681,14 @@ packages:
resolution: {integrity: sha512-qIj0G9wZbMGNLjLmg1PT6v2mE9AH2zlnADJD/2tC6E00hgmhUOfEB6greHPAfLRSufHqROIUTkw6E+M3lH0PTQ==}
engines: {node: '>= 0.4'}
+ b4a@1.9.0:
+ resolution: {integrity: sha512-dpfcF9fDNR6++cthXR67iyhgqWy9CBouAvIWhIntzBG6cvK/cnIPiZQjBwi/ZqjjBEDGfoNDtmB0kTjroOJ3pQ==}
+ peerDependencies:
+ react-native-b4a: '*'
+ peerDependenciesMeta:
+ react-native-b4a:
+ optional: true
+
balanced-match@1.0.2:
resolution: {integrity: sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw==}
@@ -2651,6 +2696,62 @@ packages:
resolution: {integrity: sha512-BLrgEcRTwX2o6gGxGOCNyMvGSp35YofuYzw9h1IMTRmKqttAZZVU67bdb9Pr2vUHA8+j3i2tJfjO6C6+4myGTA==}
engines: {node: 18 || 20 || >=22}
+ bare-addon-resolve@1.11.0:
+ resolution: {integrity: sha512-bTUBSqBjzQQzo2/iIk6KxAClZfahy38EPsJV2bPJ+p2+kS8h+hT5TE5QggpkFUrCnLL4HDTA4DDM+YiNdyol0w==}
+ peerDependencies:
+ bare-url: '*'
+ peerDependenciesMeta:
+ bare-url:
+ optional: true
+
+ bare-events@2.9.2:
+ resolution: {integrity: sha512-AIPKioV7/Y/8KfZ3AAhjPJxLLbY49S64Ym5DakZlUg75qQiTgUq9hEJoEwa4eUezPUlXRy/i5NpsKvo9jgKmoA==}
+ peerDependencies:
+ bare-abort-controller: '*'
+ peerDependenciesMeta:
+ bare-abort-controller:
+ optional: true
+
+ bare-fs@4.8.3:
+ resolution: {integrity: sha512-KQi9ShyPS/nOCtKpma7At8wbq4ORSNUmD67hsZrPcaWJcEmL8PHyS2xVWT308y8/LvHFOUTytTTDsD8dInx6bA==}
+ engines: {bare: '>=1.28.0'}
+ peerDependencies:
+ bare-buffer: '*'
+ peerDependenciesMeta:
+ bare-buffer:
+ optional: true
+
+ bare-module-resolve@1.12.6:
+ resolution: {integrity: sha512-tsoOqHmtF6VmNX4LYDBZwvQVkSmEagnbykNjK31LMxz/amyKfjCs4K976n7uJd1bvWfxbpLQ3Fy7qq8an6jR6Q==}
+ peerDependencies:
+ bare-url: '*'
+ peerDependenciesMeta:
+ bare-url:
+ optional: true
+
+ bare-path@3.1.2:
+ resolution: {integrity: sha512-ZyKbsuuqK6Ag0K8pX6V5Txq6XeJRvY+wXucnFGRjiyVYP9YWDpIQugk/b+enRYrEYBJaqLzghRQpXPMR7341Nw==}
+
+ bare-semver@1.1.0:
+ resolution: {integrity: sha512-1Hw5qJ7hXdVt3uPUqjeFTuxyvBUJauvz5A1I2jk8gzjZMHp04n//6nV9MDbG9CMw78JHY2lGV0w6s//LrASm2w==}
+
+ bare-stream@2.13.4:
+ resolution: {integrity: sha512-PcrQ8lVLbiJscNm1Kez+Yp4Gy4AHGcN1lzwjvf5NybWen7VvEgUfyfnXYJ2zNqWnzOfCb1Abq6lH8ti0syQszA==}
+ peerDependencies:
+ bare-abort-controller: '*'
+ bare-buffer: '*'
+ bare-events: '*'
+ peerDependenciesMeta:
+ bare-abort-controller:
+ optional: true
+ bare-buffer:
+ optional: true
+ bare-events:
+ optional: true
+
+ bare-url@2.5.4:
+ resolution: {integrity: sha512-Gxa7UVWBr0/edU1b+TJhn/AZvMQUj9OGspvYsaTYQrAbZA4BOTZGL3LiZxvD+CeMlDH4juwD84+eTAp/bLYW5g==}
+
base64-js@1.5.1:
resolution: {integrity: sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==}
@@ -2659,9 +2760,51 @@ packages:
engines: {node: '>=6.0.0'}
hasBin: true
+ bencode@2.0.3:
+ resolution: {integrity: sha512-D/vrAD4dLVX23NalHwb8dSvsUsxeRPO8Y7ToKA015JQYq69MLDOMkC0uGZYA/MPpltLO8rt8eqFC2j8DxjTZ/w==}
+
+ bencode@4.0.1:
+ resolution: {integrity: sha512-/bCR3FvgfB8AZk4LlowShU+nfaTNRM2vruJ9aejDa5LpgkCoRjFyEauFiZFxbpxVxQbBFapYCN42Er2dkE/Qgw==}
+ engines: {node: '>=12.20.0'}
+
+ bep53-range@2.0.1:
+ resolution: {integrity: sha512-VWN7j7sqUssWD6ROZe2m709YL3u5A1IAx1qvAbH3DAhszHt4WPvCzJLRJ+mzmZn1qWIb4mpflwdn95qezz3n8g==}
+ engines: {node: '>=12.20.0'}
+
+ bitfield@5.0.1:
+ resolution: {integrity: sha512-81mvFHCa0L1HsDdKVCTkPKZZ6KFdyXgfQNoneFT7Qk5/PZeBWdPkTFa4KttbR8uHszfjiqOJbeahLOPfCqw4cw==}
+ engines: {node: '>=20.19.0'}
+
+ bittorrent-dht@11.0.12:
+ resolution: {integrity: sha512-bFmQq874HMIBon/6KNMf/U1OG+UxJ0UzI9oPra2IRlZLuPA7VwBiA7uEpWaO2ww8G76OFjxB1p1XCOS94FuCPQ==}
+ engines: {node: '>=12.20.0'}
+
+ bittorrent-lsd@2.0.3:
+ resolution: {integrity: sha512-FE4ImhpJZo8ahzDS/Qk1Ol+3Ea1sIhCPWE2WSjiv1s8YoCSEf7JBL3bbCJW5hdVNK5ohtZ3vH5xGjZ9QWFiatQ==}
+ engines: {node: '>=12.20.0'}
+
+ bittorrent-peerid@2.0.2:
+ resolution: {integrity: sha512-V2iUvbtv31uLA8XFxbhB3f/fKTxWGaa6FI/3wSm93YAS/DS8IkeRWCyYGQ13Xbc9ajR40nsxNjGtVt7OHB3VOQ==}
+ engines: {node: '>=12.20.0'}
+
+ bittorrent-protocol@5.0.9:
+ resolution: {integrity: sha512-GIRHD1W9tFIkLbJP6OYZs6sWFJbfxaRnx4FYX1hvig/9+Ui691ZvJcQ+y6oSUmV7fh7/ehKT1lICh1/UEYsAcw==}
+ engines: {node: '>=18'}
+
+ bittorrent-tracker@11.2.3:
+ resolution: {integrity: sha512-i9pWOf1YAQWyTjDnpWwcTfEYDi5XwqD6f48pKmIHByb7x2XU86CqK+IsyMy1buKkDZ3rDKht2jIeKCZf1Dmd8g==}
+ engines: {node: '>=16.0.0'}
+ hasBin: true
+
+ bl@4.1.0:
+ resolution: {integrity: sha512-1W07cM9gS6DcLperZfFSj+bWLtaPGSOHWhPiGzXmvVJbRLdG82sH/Kn8EtW1VqWVA54AKf2h5k5BbnIbwF3h6w==}
+
blake3-wasm@2.1.5:
resolution: {integrity: sha512-F1+K8EbfOZE49dtoPtmxUQrpXaBIl3ICvasLh+nJta0xkz+9kF/7uet9fLnwKqhDrmj6g+6K3Tw9yQPUg2ka5g==}
+ block-iterator@1.1.2:
+ resolution: {integrity: sha512-yAHUP44v2K25xLPdrgVTgwtuQctlullzjczu9CoUZom5AP3g4p1R1+aWHjS1GHG9JtcSUVUnbEPiuXiW5YZ24w==}
+
bowser@2.14.1:
resolution: {integrity: sha512-tzPjzCxygAKWFOJP011oxFHs57HzIhOEracIgAePE4pqB3LikALKnSzUyU4MGs9/iCEUuHlAJTjTc5M+u7YEGg==}
@@ -2688,6 +2831,10 @@ packages:
buffer@5.6.0:
resolution: {integrity: sha512-/gDYp/UtU0eA1ys8bOs9J6a+E/KWIY+DZ+Q2WESNUA0jFRsJOc0SNUO6xJ5SGA1xueg3NL65W6s+NY5l9cunuw==}
+ bufferutil@4.1.0:
+ resolution: {integrity: sha512-ZMANVnAixE6AWWnPzlW2KpUrxhm9woycYvPOo67jWHyFowASTEd9s+QN1EIMsSDtwhIxN4sWE1jotpuDUIgyIw==}
+ engines: {node: '>=6.14.2'}
+
bytes@3.0.0:
resolution: {integrity: sha512-pMhOfFDPiv9t5jjIXkHosWmkSyQbvsgEVNkz0ERHbuLh2T/7j4Mqqpz523Fe8MVY89KC6Sh/QfS2sM+SjgFDcw==}
engines: {node: '>= 0.8'}
@@ -2696,6 +2843,9 @@ packages:
resolution: {integrity: sha512-/Nf7TyzTx6S3yRJObOAV7956r8cr2+Oj8AC5dt8wSP3BQAoeX58NoHyCU8P8zGkNXStjTSi6fzO6F0pBdcYbEg==}
engines: {node: '>= 0.8'}
+ cache-chunk-store@3.2.2:
+ resolution: {integrity: sha512-2lJdWbgHFFxcSth9s2wpId3CR3v1YC63KjP4T9WhpW7LWlY7Hiiei3QwwqzkWqlJTfR8lSy9F5kRQECeyj+yQA==}
+
call-bind-apply-helpers@1.0.2:
resolution: {integrity: sha512-Sp1ablJ0ivDkSzjcaJdxEunN5/XvksFJ2sMBFfq6x0ryhQV/2b/KwFe21cMpmHtPOSij8K99/wSfoEuTObmuMQ==}
engines: {node: '>= 0.4'}
@@ -2731,6 +2881,21 @@ packages:
resolution: {integrity: sha512-Fo07WOYGqMfCWHOzSXOt2CxDbC6skS/jO9ynEcmpANMoPrD+W1r1K6Vx7iNm+AQmETU1Xr2t+n8nzkV9t6xh3w==}
engines: {node: ^12.17.0 || ^14.13 || >=16.0.0}
+ chownr@1.1.4:
+ resolution: {integrity: sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==}
+
+ chrome-dgram@3.0.6:
+ resolution: {integrity: sha512-bqBsUuaOiXiqxXt/zA/jukNJJ4oaOtc7ciwqJpZVEaaXwwxqgI2/ZdG02vXYWUhHGziDlvGMQWk0qObgJwVYKA==}
+
+ chrome-dns@1.0.1:
+ resolution: {integrity: sha512-HqsYJgIc8ljJJOqOzLphjAs79EUuWSX3nzZi2LNkzlw3GIzAeZbaSektC8iT/tKvLqZq8yl1GJu5o6doA4TRbg==}
+
+ chrome-net@3.3.4:
+ resolution: {integrity: sha512-Jzy2EnzmE+ligqIZUsmWnck9RBXLuUy6CaKyuNMtowFG3ZvLt8d+WBJCTPEludV0DHpIKjAOlwjFmTaEdfdWCw==}
+
+ chunk-store-iterator@1.0.4:
+ resolution: {integrity: sha512-LGjzJNmk7W1mrdaBoJNztPumT2ACmgjHmI1AMm8aeGYOl4+LKaYC/yfnx27i++LiAtoe/dR+3jC8HRzb6gW4/A==}
+
class-variance-authority@0.7.1:
resolution: {integrity: sha512-Ka+9Trutv7G8M6WT6SeiRWz792K5qEqIGEGzXKhAE6xOWAY6pPH8U+9IY3oCMv6kqTmLsv7Xh/2w2RigkePMsg==}
@@ -2762,6 +2927,9 @@ packages:
color-name@1.1.4:
resolution: {integrity: sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA==}
+ compact2string@1.4.1:
+ resolution: {integrity: sha512-3D+EY5nsRhqnOwDxveBv5T8wGo4DEvYxjDtPGmdOX+gfr5gE92c2RC0w2wa+xEefm07QuVqqcF3nZJUZ92l/og==}
+
compressible@2.0.18:
resolution: {integrity: sha512-AF3r7P5dWxL8MxyITRMlORQNaOA2IkAFaTr4k7BUumjPtRpGDTZpl0Pb1XCO6JeDCBdp126Cgs9sMxqSjgYyRg==}
engines: {node: '>= 0.6'}
@@ -2784,6 +2952,17 @@ packages:
resolution: {integrity: sha512-ei8Aos7ja0weRpFzJnEA9UHJ/7XQmqglbRwnf2ATjcB9Wq874VKH9kfjjirM6UhU2/E5fFYadylyhFldcqSidQ==}
engines: {node: '>=18'}
+ cpus@1.0.3:
+ resolution: {integrity: sha512-PXHBvGLuL69u55IkLa5e5838fLhIMHxmkV4ge42a8alGyn7BtawYgI0hQ849EedvtHIOLNNH3i6eQU1BiE9SUA==}
+
+ create-torrent@6.1.3:
+ resolution: {integrity: sha512-/XARuf6/TmOLICFkUU7pf4KPL/vlIFyk2xOJjQ4np8Xzl8xLt9W7KJps1EVnZyUCMHbAm5xYfCoDrgVw3F4FxA==}
+ engines: {node: '>=12'}
+ hasBin: true
+
+ cross-fetch-ponyfill@1.0.3:
+ resolution: {integrity: sha512-uOBkDhUAGAbx/FEzNKkOfx3w57H8xReBBXoZvUnOKTI0FW0Xvrj3GrYv2iZXUqlffC1LMGfQzhmBM/ke+6eTDA==}
+
cross-spawn@7.0.6:
resolution: {integrity: sha512-uV2QOWP2nWzsy2aMp8aRibhi9dlzF5Hgh5SHaB9OiTGEyDTiJJyx0uy51QXdyWbtAHNua4XJzUKca3OzKUd3vA==}
engines: {node: '>= 8'}
@@ -2838,6 +3017,10 @@ packages:
damerau-levenshtein@1.0.8:
resolution: {integrity: sha512-sdQSFB7+llfUcQHUQO3+B8ERRj0Oa4w9POWMI/puGtuf7gFywGmkaLCElnudfTiKZV+NvHqL0ifzdrI8Ro7ESA==}
+ data-uri-to-buffer@4.0.1:
+ resolution: {integrity: sha512-0R9ikRb668HB7QDxT1vkpuUBtqc53YyAwMwGeUFKRojY/NWKvdZ+9UYtRfGmhqNbRkTSVpMbmyhXipFFv2cb/A==}
+ engines: {node: '>= 12'}
+
data-view-buffer@1.0.2:
resolution: {integrity: sha512-EmKO5V3OLXh1rtK2wgXRansaK1/mtVdTUEiEI0W8RkvgT05kfxaH29PliLnpLP73yYO6142Q72QNa8Wx/A5CqQ==}
engines: {node: '>= 0.4'}
@@ -2878,6 +3061,10 @@ packages:
decimal.js-light@2.5.1:
resolution: {integrity: sha512-qIMFpTMZmny+MMIitAB6D7iVPEorVw6YQRWkvarTkT4tBeSLLiHzcwj6q0MmYSFCiVpiqPJTJEYIrpcPzVEIvg==}
+ decompress-response@6.0.0:
+ resolution: {integrity: sha512-aW35yZM6Bb/4oJlZncMH2LCoZtJXTRxES17vE3hoRiowU2kWHaJKFkSBDnDR+cm9J+9QhXmREyIfv0pji9ejCQ==}
+ engines: {node: '>=10'}
+
deep-extend@0.6.0:
resolution: {integrity: sha512-LOHxIOaPYdHlJRtCQfDIVZtfw/ufM8+rVj649RIHzcm/vGwQRXFt6OPqIFWsm2XEMrNIEtWR64sY1LEKD2vAOA==}
engines: {node: '>=4.0.0'}
@@ -2889,6 +3076,10 @@ packages:
resolution: {integrity: sha512-3sUqbMEc77XqpdNO7FRyRog+eW3ph+GYCbj+rK+uYyRMuwsVy0rMiVtPn+QJlKFvWP/1PYpapqYn0Me2knFn+A==}
engines: {node: '>=0.10.0'}
+ default-gateway@7.2.2:
+ resolution: {integrity: sha512-AD7TrdNNPXRZIGw63dw+lnGmT4v7ggZC5NHNJgAYWm5njrwoze1q5JSAW9YuLy2tjnoLUG/r8FEB93MCh9QJPg==}
+ engines: {node: '>= 16'}
+
define-data-property@1.1.4:
resolution: {integrity: sha512-rBMvIzlpA8v6E+SJZoo++HAYqsLrkg7MSfIinMPFhmkorw7X+dOXVJQs+QT69zGkzMyfDnIMN2Wid1+NbL3T+A==}
engines: {node: '>= 0.4'}
@@ -2927,10 +3118,16 @@ packages:
emoji-regex@9.2.2:
resolution: {integrity: sha512-L18DaJsXSUk2+42pv8mLs5jJT2hqFkFE4j21wOmgbUqsZ2hL72NsUU785g9RXgo3s0ZNgVl42TiHp3ZtOv/Vyg==}
+ end-of-stream@1.4.5:
+ resolution: {integrity: sha512-ooEGc6HP26xXq/N+GCGOT0JKCLDGrq2bQUZrQ7gyrJiZANJ/8YDTxTpQBXGMn+WbIQXNVpyWymm7KYVICQnyOg==}
+
enhanced-resolve@5.21.0:
resolution: {integrity: sha512-otxSQPw4lkOZWkHpB3zaEQs6gWYEsmX4xQF68ElXC/TWvGxGMSGOvoNbaLXm6/cS/fSfHtsEdw90y20PCd+sCA==}
engines: {node: '>=10.13.0'}
+ err-code@3.0.1:
+ resolution: {integrity: sha512-GiaH0KJUewYok+eeY05IIgjtAe4Yltygk9Wqp1V5yVWLdhf0hYZchRjNIT9bb0mSwRcIusT3cx7PJUf3zEIfUA==}
+
error-stack-parser-es@1.0.5:
resolution: {integrity: sha512-5qucVt2XcuGMcEGgWI7i+yZpmpByQ8J1lHhcL7PwqCwu9FPP3VUXzT4ltHe5i2z9dePwEHcDVOAfSnHsOlCXRA==}
@@ -2980,6 +3177,9 @@ packages:
resolution: {integrity: sha512-WUj2qlxaQtO4g6Pq5c29GTcWGDyd8itL8zTlipgECz3JesAiiOKotd8JU6otB3PACgG6xkJUyVhboMS+bje/jA==}
engines: {node: '>=6'}
+ escape-html@1.0.3:
+ resolution: {integrity: sha512-NiSupZ4OeuGwr68lGIeym/ksIZMJodUGOSCZ/FSnTxcrekbvqrgdUxlJOMpijaKZVjAJrWrGs/6Jy8OMuyj9ow==}
+
escape-string-regexp@4.0.0:
resolution: {integrity: sha512-TtpcNJ3XAzx3Gq8sWRzJaVajRs0uVxA2YAkdb1jm2YkPz4G6egUFAyA3n5vtEIZefPk5Wa4UXbKuS5fKkJWdgA==}
engines: {node: '>=10'}
@@ -3109,9 +3309,16 @@ packages:
resolution: {integrity: sha512-kVscqXk4OCp68SZ0dkgEKVi6/8ij300KBWTJq32P/dYeWTSwK41WyTxalN1eRmA5Z9UU/LX9D7FWSmV9SAYx6g==}
engines: {node: '>=0.10.0'}
+ event-target-shim@5.0.1:
+ resolution: {integrity: sha512-i/2XbnSz/uxRCU6+NdVJgKWDTM427+MqYbkQzD321DuCQJUqOuJKIA0IM2+W2xtYHdKOmZ4dR6fExsd4SXL+WQ==}
+ engines: {node: '>=6'}
+
eventemitter3@4.0.7:
resolution: {integrity: sha512-8guHBZCwKnFhYdHr2ysuRWErTwhoN2X8XELRlrRwpmfeY2jjuUN4taQMsULKUVo1K4DvZl+0pgfyoysHxvmvEw==}
+ events-universal@1.0.1:
+ resolution: {integrity: sha512-LUd5euvbMLpwOF8m6ivPCbhQeSiYVNb8Vs0fQ8QjXo0JTkEHpz8pxdQf0gStltaPpw0Cca8b39KxvK9cfKRiAw==}
+
events@3.3.0:
resolution: {integrity: sha512-mQw+2fkQbALzQ7V0MY0IqdnXNOeTtP4r0lN9z7AAawCXgqea7bDii20AYrIBrFd/Hx0M2Ocz6S111CaFkUcb0Q==}
engines: {node: '>=0.8.x'}
@@ -3128,10 +3335,18 @@ packages:
resolution: {integrity: sha512-8uSpZZocAZRBAPIEINJj3Lo9HyGitllczc27Eh5YYojjMFMn8yHMDMaUHE2Jqfq05D/wucwI4JGURyXt1vchyg==}
engines: {node: '>=10'}
+ execa@7.2.0:
+ resolution: {integrity: sha512-UduyVP7TLB5IcAQl+OzLyLcS/l32W/GLg+AhHJ+ow40FOk2U3SAllPwR44v4vmdFwIWqpdwxxpQbF1n5ta9seA==}
+ engines: {node: ^14.18.0 || ^16.14.0 || >=18.0.0}
+
execa@9.6.1:
resolution: {integrity: sha512-9Be3ZoN4LmYR90tUoVu2te2BsbzHfhJyfEiAVfz7N5/zv+jduIfLrV2xdQXOHbaD6KgpGdO9PRPM1Y4Q9QkPkA==}
engines: {node: ^18.19.0 || >=20.5.0}
+ expand-template@2.0.3:
+ resolution: {integrity: sha512-XYfuKMvj4O35f/pOXLObndIRvyQ+/+6AhODh+OKWj9S9498pHHn/IMszH+gt0fBCRWMNfk1ZSp5x3AifmnI2vg==}
+ engines: {node: '>=6'}
+
fast-deep-equal@3.1.3:
resolution: {integrity: sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q==}
@@ -3139,6 +3354,9 @@ packages:
resolution: {integrity: sha512-jt2DW/aNFNwke7AUd+Z+e6pz39KO5rzdbbFCg2sGafS4mk13MI7Z8O5z9cADNn5lhGODIgLwug6TZO2ctf7kcw==}
engines: {node: '>=6.0.0'}
+ fast-fifo@1.3.2:
+ resolution: {integrity: sha512-/d9sfos4yxzpwkDkuN7k2SqFKtYNmCTzgfEpz82x34IM9/zc8KGxQoXg1liNC/izpRM/MBdt44Nmx41ZWqk+FQ==}
+
fast-glob@3.3.1:
resolution: {integrity: sha512-kNFPyjhh5cKjrUltxs+wFx+ZkbRaxxmZ+X0ZU31SOsxCEtP9VPgtq2teZw1DebupL5GmDaNQ6yKMMVcM41iqDg==}
engines: {node: '>=8.6.0'}
@@ -3149,6 +3367,9 @@ packages:
fast-levenshtein@2.0.6:
resolution: {integrity: sha512-DCXu6Ifhqcks7TZKY3Hxp3y6qphY5SJZmrWMDrKcERSOXWQdMhU9Ig/PYrzyw/ul9jOIyh0N4M0tbC5hodg8dw==}
+ fast-readable-async-iterator@2.0.0:
+ resolution: {integrity: sha512-8Sld+DuyWRIftl86ZguJxR2oXCBccOiJxrY/Rj9/7ZBynW8pYMWzIcqxFL1da+25jaWJZVa+HHX/8SsA21JdTA==}
+
fast-uri@3.1.0:
resolution: {integrity: sha512-iPeeDKJSWf4IEOasVVrknXpaBV0IApz/gp7S2bb7Z4Lljbl2MGJRqInZiUrQwV16cpzw/D3S5j5Julj/gT52AA==}
@@ -3164,6 +3385,10 @@ packages:
picomatch:
optional: true
+ fetch-blob@3.2.0:
+ resolution: {integrity: sha512-7yAQpD2UMJzLi1Dqv7qFYnPbaPx7ZfFK6PiIxQ4PfkGPyNyl2Ugx+a/umUonmKqjhM4DnfbMvdX6otXq83soQQ==}
+ engines: {node: ^12.20 || >= 14.13}
+
figures@6.1.0:
resolution: {integrity: sha512-d+l3qxjSesT4V7v2fh+QnmFnUWv9lSpjarhShNTgBOfA0ttejbQUAlHLitbjkoRiDulW0OPoQPYIGhIC8ohejg==}
engines: {node: '>=18'}
@@ -3172,6 +3397,10 @@ packages:
resolution: {integrity: sha512-XXTUwCvisa5oacNGRP9SfNtYBNAMi+RPwBFmblZEF7N7swHYQS6/Zfk7SRwx4D5j3CH211YNRco1DEMNVfZCnQ==}
engines: {node: '>=16.0.0'}
+ filename-reserved-regex@3.0.0:
+ resolution: {integrity: sha512-hn4cQfU6GOT/7cFHXBqeBg2TbrMBgdD0kcjLhvSQYYwm3s4B6cjvBfb7nBALJLAXqmU5xajSa7X2NnUud/VCdw==}
+ engines: {node: ^12.20.0 || ^14.13.1 || >=16.0.0}
+
fill-range@7.1.1:
resolution: {integrity: sha512-YsGpe3WHLK8ZYi4tWDg2Jy3ebRz2rXowDxnld4bkQB00cc/1Zw9AWnC0i9ztDJitivtQvaI9KaLyKrc+hBW0yg==}
engines: {node: '>=8'}
@@ -3194,10 +3423,30 @@ packages:
resolution: {integrity: sha512-dKx12eRCVIzqCxFGplyFKJMPvLEWgmNtUrpTiJIR5u97zEhRG8ySrtboPHZXx7daLxQVrl643cTzbab2tkQjxg==}
engines: {node: '>= 0.4'}
+ formdata-polyfill@4.0.10:
+ resolution: {integrity: sha512-buewHzMvYL29jdeQTVILecSaZKnt/RJWjoZCF5OW60Z67/GmSLBkOFM7qh1PI3zFNtJbaZL5eQu1vLfazOwj4g==}
+ engines: {node: '>=12.20.0'}
+
+ freelist@1.0.3:
+ resolution: {integrity: sha512-Ji7fEnMdZDGbS5oXElpRJsn9jPvBR8h/037D3bzreNmS8809cISq/2D9//JbA/TaZmkkN8cmecXwmQHmM+NHhg==}
+
+ fs-chunk-store@5.0.1:
+ resolution: {integrity: sha512-bhD1YfAzdOugxu4VepYjxvzZwik9Qunn69Ogj47DZmogD6h/VkD/4vArNRoQ+kPYYOFqex3c9Ob2iAOkNYdkgw==}
+ engines: {node: '>=12.20.0'}
+
+ fs-constants@1.0.0:
+ resolution: {integrity: sha512-y6OAwoSIf7FyjMIv94u+b5rdheZEjzR63GTyZJm5qh4Bi+2YgwLCcI/fPFZkL5PSixOt6ZNKm+w+Hfp/Bciwow==}
+
fs-extra@11.3.4:
resolution: {integrity: sha512-CTXd6rk/M3/ULNQj8FBqBWHYBVYybQ3VPBw0xGKFe3tuH7ytT6ACnvzpIQ3UZtB8yvUKC2cXn1a+x+5EVQLovA==}
engines: {node: '>=14.14'}
+ fs-native-extensions@1.5.1:
+ resolution: {integrity: sha512-abjiHKkYdcH5M9ikBEJb0MKb/fEpPtZx/yfLHzTptvUAoiFayX0tIe0BTLBU4SAoRyjZLzA0dP1Rn2p0+QRyVg==}
+
+ fsa-chunk-store@1.3.0:
+ resolution: {integrity: sha512-0WCfuxqqSB6Tz/g7Ar/nwAxMoigXaIXuvfrnLIEFYIA9uc6w9eNaHuBGzU1X3lyM4cpLKCOTUmKAA/gCiTvzMQ==}
+
fsevents@2.3.2:
resolution: {integrity: sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==}
engines: {node: ^8.16.0 || ^10.6.0 || >=11.0.0}
@@ -3238,6 +3487,10 @@ packages:
resolution: {integrity: sha512-sTSfBjoXBp89JvIKIefqw7U2CCebsc74kiY6awiGogKtoSGbgjYE/G/+l9sF3MWFPNc9IcoOC4ODfKHfxFmp0g==}
engines: {node: '>= 0.4'}
+ get-stdin@10.0.0:
+ resolution: {integrity: sha512-eWSePJ4zXFdqz+/Lyfopob4rIcoF/U2XfE8nJc7iZV6lnebWc9k7DoQQpX+2a9jc0AOvBsXvbe5YkjXl/MHbpg==}
+ engines: {node: '>=20'}
+
get-stream@6.0.1:
resolution: {integrity: sha512-ts6Wi+2j3jQjqi70w5AlN8DFnkSwC+MqmxEzdEALB2qXZYV3X/b1CTfgPLGJNMeAWxdPfU8FO1ms3NUfaHCPYg==}
engines: {node: '>=10'}
@@ -3253,6 +3506,9 @@ packages:
get-tsconfig@4.14.0:
resolution: {integrity: sha512-yTb+8DXzDREzgvYmh6s9vHsSVCHeC0G3PI5bEXNBHtmshPnO+S5O7qgLEOn0I5QvMy6kpZN8K1NKGyilLb93wA==}
+ github-from-package@0.0.0:
+ resolution: {integrity: sha512-SyHy3T1v2NUXn29OsWdxmK6RwHD+vkj3v8en8AOBZ1wBQ/hCAQ5bAQTD02kW4W9tUp/3Qh6J8r9EvntiyCmOOw==}
+
glob-parent@5.1.2:
resolution: {integrity: sha512-AOIgSQCepiJYwP3ARnGx+5VnTu2HBYdzbGP45eLw1vr3zB3vZLeyed1sC9hnbcOc9/SrMyM5RPQrkGz4aS9Zow==}
engines: {node: '>= 6'}
@@ -3316,10 +3572,17 @@ packages:
hls.js@1.6.16:
resolution: {integrity: sha512-VSIRpLfRwlAAdGL4wiTucx2ScRipo0ed1FBatWkyt832jC4CReKstga6yIhYVwGu9LOBjuX9wzmRMeQdBJtzEA==}
+ http-parser-js@0.4.13:
+ resolution: {integrity: sha512-u8u5ZaG0Tr/VvHlucK2ufMuOp4/5bvwgneXle+y228K5rMbJOlVjThONcaAw3ikAy8b2OO9RfEucdMHFz3UWMA==}
+
human-signals@2.1.0:
resolution: {integrity: sha512-B4FFZ6q/T2jhhksgkbEW3HBvWIfDW85snkQgawt07S7J5QXTk6BkNV+0yAeZrM5QpMAdYlocGoljn0sJ/WQkFw==}
engines: {node: '>=10.17.0'}
+ human-signals@4.3.1:
+ resolution: {integrity: sha512-nZXjEF2nbo7lIw3mgYjItAfgQXog3OjJogSbKa2CQIIvSGWcKgeJnQlNXip6NglNzYH45nSRiEVimMvYL8DDqQ==}
+ engines: {node: '>=14.18.0'}
+
human-signals@8.0.1:
resolution: {integrity: sha512-eKCa6bwnJhvxj14kZk5NCPc6Hb6BdsU9DZcOnmQKSnO1VKrfV0zCvtttPZUsBvjmNDn8rpcJfpwSYnHBjc95MQ==}
engines: {node: '>=18.18.0'}
@@ -3335,6 +3598,9 @@ packages:
resolution: {integrity: sha512-Hs59xBNfUIunMFgWAbGX5cq6893IbWg4KnrjbYwX3tx0ztorVgTDA6B2sxf8ejHJ4wz8BqGUMYlnzNBer5NvGg==}
engines: {node: '>= 4'}
+ immediate-chunk-store@2.2.0:
+ resolution: {integrity: sha512-1bHBna0hCa6arRXicu91IiL9RvvkbNYLVq+mzWdaLGZC3hXvX4doh8e1dLhMKez5siu63CYgO5NrGJbRX5lbPA==}
+
import-fresh@3.3.1:
resolution: {integrity: sha512-TR3KfrTZTYLPB6jUjfx6MF9WcWrHL9su5TObK4ZkYgBdWKPOFoSoQIdEuTuR82pmtxH2spWG9h6etwfr1pLBqQ==}
engines: {node: '>=6'}
@@ -3357,6 +3623,25 @@ packages:
resolution: {integrity: sha512-5Hh7Y1wQbvY5ooGgPbDaL5iYLAPzMTUrjMulskHLH6wnv/A+1q5rgEaiuqEjB+oxGXIVZs1FF+R/KPN3ZSQYYg==}
engines: {node: '>=12'}
+ ip-address@10.7.3:
+ resolution: {integrity: sha512-A1kdq/tSb5QjvKvAMgIoEvDBIgL7qaqVP/jkvSwYYRZ9iEzvPpopxp2wQfu3SuZRHtpHNxMn8Fs0bS+gf5Xmwg==}
+ engines: {node: '>= 12'}
+
+ ip-set@3.0.0:
+ resolution: {integrity: sha512-EkKzllMFYKMHOoeNxM8v10IJM2n9K6L3ruaQx4A/5RrkOInV8cjp9IwkVU0/cpzwwkq1zGNudrAdCjSR97Rf5w==}
+ engines: {node: '>=18'}
+
+ ip@2.0.1:
+ resolution: {integrity: sha512-lJUL9imLTNi1ZfXT+DU6rBBdbiKGBuay9B6xGSPVjUeQwaH1RIGqef8RZkUtHioLmSNpPR5M4HVKJGm1j8FWVQ==}
+
+ ipaddr.js@2.0.1:
+ resolution: {integrity: sha512-1qTgH9NG+IIJ4yfKs2e6Pp1bZg8wbDbKHT21HrLIeYBTRLgMYKnMTPAuI3Lcs61nfx5h1xlXnbJtH1kX5/d/ng==}
+ engines: {node: '>= 10'}
+
+ ipaddr.js@2.5.0:
+ resolution: {integrity: sha512-aq+t5NAc+cS6rZQQVWC2x98CPqGtKKTMDd4Gaodv0wShnItdKg/51djkGJ1hqH+Oy0ivDftCbSLCQob8zso01w==}
+ engines: {node: '>= 10'}
+
is-array-buffer@3.0.5:
resolution: {integrity: sha512-DDfANUiiG2wC1qawP66qlTugJeL5HyzMpfr8lLK+jMQirGzNod0B12cFB/9q838Ru27sBwfw78/rdoU7RERz6A==}
engines: {node: '>= 0.4'}
@@ -3401,6 +3686,9 @@ packages:
resolution: {integrity: sha512-SbKbANkN603Vi4jEZv49LeVJMn4yGwsbzZworEoyEiutsN3nJYdbO36zfhGJ6QEDpOZIFkDtnq5JRxmvl3jsoQ==}
engines: {node: '>=0.10.0'}
+ is-file@1.0.0:
+ resolution: {integrity: sha512-ZGMuc+xA8mRnrXtmtf2l/EkIW2zaD2LSBWlaOVEF6yH4RTndHob65V4SwWWdtGKVthQfXPVKsXqw4TDUjbVxVQ==}
+
is-finalizationregistry@1.1.1:
resolution: {integrity: sha512-1pC6N8qWJbWoPtEjgcL2xyhQOP491EQjeUo3qTKcmV8YSDDJrOepfG8pcC7h/QgnQHYSv0mJ3Z/ZWxmatVrysg==}
engines: {node: '>= 0.4'}
@@ -3457,6 +3745,10 @@ packages:
resolution: {integrity: sha512-hFoiJiTl63nn+kstHGBtewWSKnQLpyb155KHheA1l39uvtO9nWIop1p3udqPcUd/xbF1VLMO4n7OI6p7RbngDg==}
engines: {node: '>=8'}
+ is-stream@3.0.0:
+ resolution: {integrity: sha512-LnQR4bZ9IADDRSkvpqMGvt/tEJWclzklNgSw48V5EAaAeDd6qGvN8ei6k5p0tvxSR171VmGyHuTiAOfxAbr8kA==}
+ engines: {node: ^12.20.0 || ^14.13.1 || >=16.0.0}
+
is-stream@4.0.1:
resolution: {integrity: sha512-Dnz92NInDqYckGEUJv689RbRiTSEHCQ7wOVeALbkOz999YpqT46yMRIGtSNl2iCL1waAZSx40+h59NV/EwzV/A==}
engines: {node: '>=18'}
@@ -3507,6 +3799,9 @@ packages:
resolution: {integrity: sha512-ekilCSN1jwRvIbgeg/57YFh8qQDNbwDb9xT/qu2DAHbFFZUicIl4ygVaAvzveMhMVr3LnpSKTNnwt8PoOfmKhQ==}
hasBin: true
+ join-async-iterator@1.1.1:
+ resolution: {integrity: sha512-ATse+nuNeKZ9K1y27LKdvPe/GCe9R/u9dw9vI248e+vILeRK3IcJP4JUPAlSmKRCDK0cKhEwfmiw4Skqx7UnGQ==}
+
jose@6.2.3:
resolution: {integrity: sha512-YYVDInQKFJfR/xa3ojUTl8c2KoTwiL1R5Wg9YCydwH0x0B9grbzlg5HC7mMjCtUJjbQ/YnGEZIhI5tCgfTb4Hw==}
@@ -3550,6 +3845,19 @@ packages:
resolution: {integrity: sha512-ZZow9HBI5O6EPgSJLUb8n2NKgmVWTwCvHGwFuJlMjvLFqlGG6pjirPhtdsseaLZjSibD8eegzmYpUZwoIlj2cQ==}
engines: {node: '>=4.0'}
+ junk@4.0.1:
+ resolution: {integrity: sha512-Qush0uP+G8ZScpGMZvHUiRfI0YBWuB3gVBYlI0v0vvOJt5FLicco+IkP0a50LqTTQhmts/m6tP5SWE+USyIvcQ==}
+ engines: {node: '>=12.20'}
+
+ k-bucket@5.1.0:
+ resolution: {integrity: sha512-Fac7iINEovXIWU20GPnOMLUbjctiS+cnmyjC4zAUgvs3XPf1vo9akfCHkigftSic/jiKqKl+KA3a/vFcJbHyCg==}
+
+ k-rpc-socket@1.11.1:
+ resolution: {integrity: sha512-8xtA8oqbZ6v1Niryp2/g4GxW16EQh5MvrUylQoOG+zcrDff5CKttON2XUXvMwlIHq4/2zfPVFiinAccJ+WhxoA==}
+
+ k-rpc@5.1.0:
+ resolution: {integrity: sha512-FGc+n70Hcjoa/X2JTwP+jMIOpBz+pkRffHnSl9yrYiwUxg3FIgD50+u1ePfJUOnRCnx6pbjmVk5aAeB1wIijuQ==}
+
keyv@4.5.4:
resolution: {integrity: sha512-oxVHkHR/EJf2CNXnWxRLW6mg7JyCCUcG0DtEGmL2ctUo1PNTin1PUil+r/+4r5MpVgC/fn1kjsx7mjSujKqIpw==}
@@ -3564,6 +3872,9 @@ packages:
resolution: {integrity: sha512-MbjN408fEndfiQXbFQ1vnd+1NoLDsnQW41410oQBXiyXDMYH5z505juWa4KUE1LqxRC7DgOgZDbKLxHIwm27hA==}
engines: {node: '>=0.10'}
+ last-one-wins@1.0.4:
+ resolution: {integrity: sha512-t+KLJFkHPQk8lfN6WBOiGkiUXoub+gnb2XTYI2P3aiISL+94xgZ1vgz1SXN/N4hthuOoLXarXfBZPUruyjQtfA==}
+
levn@0.4.1:
resolution: {integrity: sha512-+bT2uH4E5LGE7h/n3evcS/sQlJXCpIp6ym8OWJ5eV6+67Dsql/LaaT7qJBAt2rzfoa/5QBGBhxDix1dMt2kQKQ==}
engines: {node: '>= 0.8.0'}
@@ -3642,10 +3953,17 @@ packages:
resolution: {integrity: sha512-NXYBzinNrblfraPGyrbPoD19C1h9lfI/1mzgWYvXUTe414Gz/X1FD2XBZSZM7rRTrMA8JL3OtAaGifrIKhQ5yQ==}
engines: {node: '>= 12.0.0'}
+ limiter@1.1.5:
+ resolution: {integrity: sha512-FWWMIEOxz3GwUI4Ts/IvgVy6LPvoMPgjMdQ185nN6psJyBJ4yOpzqm695/h5umdLJg2vW3GR5iG11MAkR2AzJA==}
+
lmdb@3.5.4:
resolution: {integrity: sha512-9FKQA6G1MMtqNxfxvSBNXD/axeG2QRjYbNh0/ykRL5xYcRbCm2vXq7B9bhc7nSuKdHzr8/BHIwfPuYYH1UsXXw==}
hasBin: true
+ load-ip-set@3.0.2:
+ resolution: {integrity: sha512-UD1GM3CLlkC3b0gAKIxd+6SFJb1WQttWyYhwvjdWjGpJKzu32HnaSMfWtUtVgRtFY+K5vgrvecuVQLRxx5Ojag==}
+ engines: {node: '>=12.20.0'}
+
load-script@1.0.0:
resolution: {integrity: sha512-kPEjMFtZvwL9TaZo0uZ2ml+Ye9HUMmPwbYRJ324qF9tqMejwykJ5ggTyvzmrbBeapCAbk98BSbTeovHEEP1uCA==}
@@ -3666,6 +3984,14 @@ packages:
lru-cache@5.1.1:
resolution: {integrity: sha512-KpNARQA3Iwv+jTA0utUVVbrh+Jlrr1Fv0e56GGzAFOXN7dk/FviaDW8LHmK52DlcH4WP2n6gI8vN1aesBFgo9w==}
+ lru@3.1.0:
+ resolution: {integrity: sha512-5OUtoiVIGU4VXBOshidmtOsvBIvcQR6FD/RzWSvaeHyxCGB+PCUCu+52lqMfdc0h/2CLvHhZS4TwUmMQrrMbBQ==}
+ engines: {node: '>= 0.4.0'}
+
+ lt_donthave@2.0.7:
+ resolution: {integrity: sha512-FdfetmgzUr0IC2ZXHaOhAMEsvw8BFo7kDM7JzfYsJ5pO5+xzjfZIPdLSQWE4+/UTlQA7D8IU5ZJ0LTuLE4BRBQ==}
+ engines: {node: '>=12.20.0'}
+
lucide-react@1.16.0:
resolution: {integrity: sha512-dYwyPzb4MEKpGUmNYk3WKWPnMrHs3FKM+q94kAnJrcDIqqn1hq2xY8scaS2ovsOCM5D51ey2gaRG3PBb1vgoYQ==}
peerDependencies:
@@ -3674,6 +4000,10 @@ packages:
magic-string@0.30.21:
resolution: {integrity: sha512-vd2F4YUyEXKGcLHoq+TEyCjxueSeHnFxyyjNp80yg0XV4vUhnDer/lvvlqM/arB5bXQN5K2/3oinyCRyx8T2CQ==}
+ magnet-uri@7.0.10:
+ resolution: {integrity: sha512-PjS/xWnvq9RK6P/8+dWG7ka3mFGGt/R9xiq5A+P8a8MpE40Dw6yNyTdjCFuJHS3apn6p5bJUCzMSceW3VxezRA==}
+ engines: {node: '>=12.20.0'}
+
markdown-to-jsx@7.7.17:
resolution: {integrity: sha512-7mG/1feQ0TX5I7YyMZVDgCC/y2I3CiEhIRQIhyov9nGBP5eoVrOXXHuL5ZP8GRfxVZKRiXWJgwXkb9It+nQZfQ==}
engines: {node: '>= 10'}
@@ -3690,6 +4020,9 @@ packages:
memoize-one@5.2.1:
resolution: {integrity: sha512-zYiwtZUcYyXKo/np96AGZAckk+FWWsUdJ3cHGGmld7+AhvcWmQyGCYUh1hc4Q/pkOhb65dQR/pqCyK0cOaHz4Q==}
+ memory-chunk-store@1.3.5:
+ resolution: {integrity: sha512-E1Xc1U4ifk/FkC2ZsWhCaW1xg9HbE/OBmQTLe2Tr9c27YPSLbW7kw1cnb3kQWD1rDtErFJHa7mB9EVrs7aTx9g==}
+
merge-stream@2.0.0:
resolution: {integrity: sha512-abv/qOcuPfk3URPfDzmZU1LKmuw8kT+0nIHvKrKgFrwifol/doWcdA4ZqsWQ8ENrFKkd67Mfpo/LovbIUsbt3w==}
@@ -3713,10 +4046,23 @@ packages:
resolution: {integrity: sha512-lc/aahn+t4/SWV/qcmumYjymLsWfN3ELhpmVuUFjgsORruuZPVSwAQryq+HHGvO/SI2KVX26bx+En+zhM8g8hQ==}
engines: {node: '>= 0.6'}
+ mime@3.0.0:
+ resolution: {integrity: sha512-jSCU7/VB1loIWBZe14aEYHU/+1UMEHoaO7qxCOVJOw9GgH72VAWppxNcjU+x9a2k3GSIBXNKxXQFqRvvZ7vr3A==}
+ engines: {node: '>=10.0.0'}
+ hasBin: true
+
mimic-fn@2.1.0:
resolution: {integrity: sha512-OqbOk5oEQeAZ8WXWydlu9HJjz9WVdEIvamMCcXmuqUYjTknH/sqsWvhQ3vgwKFRR1HpjvNBKQ37nbJgYzGqGcg==}
engines: {node: '>=6'}
+ mimic-fn@4.0.0:
+ resolution: {integrity: sha512-vqiC06CuhBTUdZH+RYl8sFrL096vA45Ok5ISO6sE/Mr1jRbGH4Csnhi8f3wKVl7x8mO4Au7Ir9D3Oyv1VYMFJw==}
+ engines: {node: '>=12'}
+
+ mimic-response@3.1.0:
+ resolution: {integrity: sha512-z0yWI+4FDrrweS8Zmt4Ej5HdJmky15+L2e6Wgn3+iK5fWzb6T3fhNFq2+MeTRb064c6Wr4N/wv0DzQTjNzHNGQ==}
+ engines: {node: '>=10'}
+
miniflare@5.20261001.0-alpha:
resolution: {integrity: sha512-GaimS5mSIOMyvd16ga+e1/QkI8cmp3z35QPHFex0DktqNQf2RVZQwMPQXdyoUreUiFP/0Gpe2MoQInTyMAD5xA==}
engines: {node: '>=22.0.0'}
@@ -3731,6 +4077,9 @@ packages:
minimist@1.2.8:
resolution: {integrity: sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==}
+ mkdirp-classic@0.5.3:
+ resolution: {integrity: sha512-gKLcREMhtuZRwRAfqP3RFW+TK4JqApVBtOIftVgjuABpAtpxhPGaDcfvbhNvD0B8iD1oUr/txX35NjcaY6Ns/A==}
+
ms@2.0.0:
resolution: {integrity: sha512-Tpp60P6IUJDTuOq/5Z8cdskzJujfwqfOTkrwIwj7IRISpnkJnT6SyJ4PCPnGMoFjC9ddhal5KVIYtAt97ix05A==}
@@ -3749,6 +4098,12 @@ packages:
engines: {node: ^10 || ^12 || ^13.7 || ^14 || >=15.0.1}
hasBin: true
+ napi-build-utils@2.0.0:
+ resolution: {integrity: sha512-GEbrYkbfF7MoNaoh2iGG84Mnf/WZfB0GdGEsM8wz7Expx/LlWf5U8t9nvJKXSp3qr5IsEbK04cBGhol/KwOsWA==}
+
+ napi-macros@2.2.2:
+ resolution: {integrity: sha512-hmEVtAGYzVQpCKdbQea4skABsdXW4RUh5t5mJ2zzqowJS2OyXZTU1KhDVFhx+NlWZ4ap9mqR9TcDO3LTTttd+g==}
+
napi-postinstall@0.3.4:
resolution: {integrity: sha512-PHI5f1O0EP5xJ9gQmFGMS6IZcrVvTjpXjz7Na41gTE7eE2hK11lg04CECCYEEjdc17EV4DO+fkGEtt7TpTaTiQ==}
engines: {node: ^12.20.0 || ^14.18.0 || >=16.0.0}
@@ -3761,6 +4116,10 @@ packages:
resolution: {integrity: sha512-myRT3DiWPHqho5PrJaIRyaMv2kgYf0mUVgBNOYMuCH5Ki1yEiQaf/ZJuQ62nvpc44wL5WDbTX7yGJi1Neevw8w==}
engines: {node: '>= 0.6'}
+ netmask@2.1.1:
+ resolution: {integrity: sha512-eonl3sLUha+S1GzTPxychyhnUzKyeQkZ7jLjKrBagJgPla13F+uQ71HgpFefyHgqrjEbCPkDArxYsjY8/+gLKA==}
+ engines: {node: '>= 0.4.0'}
+
next@16.2.3:
resolution: {integrity: sha512-9V3zV4oZFza3PVev5/poB9g0dEafVcgNyQ8eTRop8GvxZjV2G15FC5ARuG1eFD42QgeYkzJBJzHghNP8Ad9xtA==}
engines: {node: '>=20.9.0'}
@@ -3782,17 +4141,38 @@ packages:
sass:
optional: true
+ node-abi@3.98.0:
+ resolution: {integrity: sha512-Llh8noNw7s15m+rmIrZnZQbe4gLyasL9loT74rC92PkmwWKeEedKvc7o6pjp8B3MhrYT/Cl846NOYZbz1wmddw==}
+ engines: {node: '>=10'}
+
node-addon-api@6.1.0:
resolution: {integrity: sha512-+eawOlIgy680F0kBzPUNFhMZGtJ1YmqM6l4+Crf4IkImjYrO/mqPwRMh352g23uIaQKFItcQ64I7KMaJxHgAVA==}
+ node-datachannel@0.32.3:
+ resolution: {integrity: sha512-Aok1ZhLsll472lRefgWYuWJ0070jh0ecHravTdRyZEmoESumebMEQV8Y+poBwSW2ZbEwAokAOGsK5Cu8pDDT2g==}
+ engines: {node: '>=18.20.0'}
+
+ node-domexception@1.0.0:
+ resolution: {integrity: sha512-/jKZoMpw0F8GRwl4/eLROPA3cfcXtLApP0QzLmUT/HuPCZWyB7IY9ZrMeKw2O/nFIqPQB3PVM9aYm0F312AXDQ==}
+ engines: {node: '>=10.5.0'}
+ deprecated: Use your platform's native DOMException instead
+
node-exports-info@1.6.0:
resolution: {integrity: sha512-pyFS63ptit/P5WqUkt+UUfe+4oevH+bFeIiPPdfb0pFeYEu/1ELnJu5l+5EcTKYL5M7zaAa7S8ddywgXypqKCw==}
engines: {node: '>= 0.4'}
+ node-fetch@3.3.2:
+ resolution: {integrity: sha512-dRB78srN/l6gqWulah9SrxeYnxeddIG30+GOqK/9OlLVyLg3HPnr6SqOWTWOXKRwC2eGYCkZ59NNuSgvSrpgOA==}
+ engines: {node: ^12.20.0 || ^14.13.1 || >=16.0.0}
+
node-gyp-build-optional-packages@5.2.2:
resolution: {integrity: sha512-s+w+rBWnpTMwSFbaE0UXsRlg7hU4FjekKU4eyAih5T8nJuNZT1nNsskXpxmeqSK9UzkBl6UgRlnKc8hz8IEqOw==}
hasBin: true
+ node-gyp-build@4.8.4:
+ resolution: {integrity: sha512-LA4ZjwlnUblHVgq0oBF3Jl/6h/Nvs5fzBLwdEF4nuxnFdsfajde4WfxtJr3CaiH+F6ewcIB/q4jQ4UzPyid+CQ==}
+ hasBin: true
+
node-releases@2.0.38:
resolution: {integrity: sha512-3qT/88Y3FbH/Kx4szpQQ4HzUbVrHPKTLVpVocKiLfoYvw9XSGOX2FmD2d6DrXbVYyAQTF2HeF6My8jmzx7/CRw==}
@@ -3800,6 +4180,10 @@ packages:
resolution: {integrity: sha512-S48WzZW777zhNIrn7gxOlISNAqi9ZC/uQFnRdbeIHhZhCA6UqpkOT8T1G7BvfdgP4Er8gF4sUbaS0i7QvIfCWw==}
engines: {node: '>=8'}
+ npm-run-path@5.3.0:
+ resolution: {integrity: sha512-ppwTtiJZq0O/ai0z7yfudtBpWIoxM8yE6nHi1X47eFR2EWORqfbu6CnPlNsjeN683eT0qG6H/Pyf9fCcvjnnnQ==}
+ engines: {node: ^12.20.0 || ^14.13.1 || >=16.0.0}
+
npm-run-path@6.0.0:
resolution: {integrity: sha512-9qny7Z9DsQU8Ou39ERsPU4OZQlSTP47ShQzuKZ6PRXpYLtIFgl/DEBYEXKlvcEa+9tHVcK8CF81Y2V72qaZhWA==}
engines: {node: '>=18'}
@@ -3840,10 +4224,17 @@ packages:
resolution: {integrity: sha512-737ZY3yNnXy37FHkQxPzt4UZ2UWPWiCZWLvFZ4fu5cueciegX0zGPnrlY6bwRg4FdQOe9YU8MkmJwGhoMybl8A==}
engines: {node: '>= 0.8'}
+ once@1.4.0:
+ resolution: {integrity: sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w==}
+
onetime@5.1.2:
resolution: {integrity: sha512-kbpaSSGJTWdAY5KPVeMOKXSrPtr8C8C7wodJbcsd51jRnmD+GZu8Y0VoU6Dm5Z4vWr0Ig/1NKuWRKf7j5aaYSg==}
engines: {node: '>=6'}
+ onetime@6.0.0:
+ resolution: {integrity: sha512-1FlR+gjXK7X+AsAHso35MnyN5KqGwJRi/31ft6x0M194ht7S+rWAvd7PHss9xSKMzE0asv1pyIHaJYq+BbacAQ==}
+ engines: {node: '>=12'}
+
optionator@0.9.4:
resolution: {integrity: sha512-6IpQ7mKUxRcZNLIObR0hz7lxsapSSIYNZJwXPGeF0mTVqGKFIXj1DQcMoT22S3ROcLyY/rz0PWaWZ9ayWmad9g==}
engines: {node: '>= 0.8.0'}
@@ -3875,6 +4266,11 @@ packages:
resolution: {integrity: sha512-TXfryirbmq34y8QBwgqCVLi+8oA3oWx2eAnSn62ITyEhEYaWRlVZ2DvMM9eZbMs/RfxPu/PK/aBLyGj4IrqMHw==}
engines: {node: '>=18'}
+ parse-torrent@11.0.24:
+ resolution: {integrity: sha512-6nNWYiJq6egD8pMqZYL1mfnAly9ZH32DHkAYgZuAfwWij52ce7e3BtrWGVw2q1Bpb5Us8YQVGN7ML/47z/bVIA==}
+ engines: {node: '>=12.20.0'}
+ hasBin: true
+
path-exists@4.0.0:
resolution: {integrity: sha512-ak9Qy5Q7jYb2Wwcey5Fpvg2KoAc/ZIhLSLOSBmRmygPsGwkVVt0fZa0qrtMz+m6tJTAHfZQ8FnmB4MG4LWy7/w==}
engines: {node: '>=8'}
@@ -3913,6 +4309,9 @@ packages:
resolution: {integrity: sha512-QP88BAKvMam/3NxH6vj2o21R6MjxZUAd6nlwAS/pnGvN9IVLocLHxGYIzFhg6fUQ+5th6P4dv4eW9jX3DSIj7A==}
engines: {node: '>=12'}
+ piece-length@2.0.1:
+ resolution: {integrity: sha512-dBILiDmm43y0JPISWEmVGKBETQjwJe6mSU9GND+P9KW0SJGUwoU/odyH1nbalOP9i8WSYuqf1lQnaj92Bhw+Ug==}
+
pkce-challenge@5.0.1:
resolution: {integrity: sha512-wQ0b/W4Fr01qtpHlqSqspcj3EhBvimsdh0KlHhH8HRZnMsEa0ea2fTULOXOS9ccQr3om+GcGRk4e+isrZWV8qQ==}
engines: {node: '>=16.20.0'}
@@ -3939,6 +4338,12 @@ packages:
resolution: {integrity: sha512-W62t/Se6rA0Az3DfCL0AqJwXuKwBeYg6nOaIgzP+xZ7N5BFCI7DYi1qs6ygUYT6rvfi6t9k65UMLJC+PHZpDAA==}
engines: {node: ^10 || ^12 || >=14}
+ prebuild-install@7.1.3:
+ resolution: {integrity: sha512-8Mf2cbV7x1cXPUILADGI3wuhfqWvtiLA1iclTDbFRZkgRQS0NqsPZphna9V+HyTEadheuPmjaJMsbzKQFOzLug==}
+ engines: {node: '>=10'}
+ deprecated: No longer maintained. Please contact the author of the relevant native addon; alternatives are available.
+ hasBin: true
+
prelude-ls@1.2.1:
resolution: {integrity: sha512-vkcDPrRZo1QZLbn5RLGPpg/WmIQ65qoWWhcGKf/b5eplkkarX0m9z8ppCat4mlOqUsWpyNuYgO3VRyrYHSzX5g==}
engines: {node: '>= 0.8.0'}
@@ -3950,6 +4355,9 @@ packages:
prop-types@15.8.1:
resolution: {integrity: sha512-oj87CgZICdulUohogVAR7AjlC0327U4el4L6eAvOqCeudMDVU0NThNaV+b9Df4dXgSP1gXMTnPdhfe/2qDH5cg==}
+ pump@3.0.4:
+ resolution: {integrity: sha512-VS7sjc6KR7e1ukRFhQSY5LM2uBWAUPiOPa/A3mkKmiMwSmRFUITt0xuj+/lesgnCv+dPIEYlkzrcyXgquIHMcA==}
+
punycode@2.3.1:
resolution: {integrity: sha512-vYt7UD1U9Wg6138shLtLOvdAu+8DsC/ilFtEVHcH+wydcSpNE20AfSOduf6MkRFahL5FY7X1oU7nKVZFtfq8Fg==}
engines: {node: '>=6'}
@@ -3957,6 +4365,9 @@ packages:
queue-microtask@1.2.3:
resolution: {integrity: sha512-NuaNSa6flKT5JaSYQzJok04JzTL1CA6aGhv5rfLW3PgqA+M2ChpZQnAC8h8i4ZFkBS8X5RqkDBHA7r4hej3K9A==}
+ queue-tick@1.0.1:
+ resolution: {integrity: sha512-kJt5qhMxoszgU/62PLP1CJytzd2NKetjSRnyuj31fDd3Rlcz3fzlFdFLD1SItunPwyqEOkca6GbV612BWfaBag==}
+
radix-ui@1.6.0:
resolution: {integrity: sha512-EUEC70O03EgxWMP5aoqfBZ6iLC5bczFagGy7zhSYRt8o5DP7IWNiP3ywetse3L9b8843ExB0OGWZvgbYVJuNeg==}
peerDependencies:
@@ -3970,10 +4381,26 @@ packages:
'@types/react-dom':
optional: true
+ random-access-file@4.1.2:
+ resolution: {integrity: sha512-GQM6R78DceZDcQod8KxlDFwXIiUvlvuy1EOzxTDsjuDjW5NlnlZi0MOk6iI4itAj/2vcvdqcEExYbVpC/dJcEw==}
+
+ random-access-storage@3.0.2:
+ resolution: {integrity: sha512-Es9maUyWdJXWKckKy9s1+vT+DEgAt+PBb9lxPaake/0EDUsHehloKGv9v1zimS2V3gpFAcQXubvc1Rgci2sDPQ==}
+
+ random-iterate@1.0.1:
+ resolution: {integrity: sha512-Jdsdnezu913Ot8qgKgSgs63XkAjEsnMcS1z+cC6D6TNXsUXsMxy0RpclF2pzGZTEiTXL9BiArdGTEexcv4nqcA==}
+
+ randombytes@2.1.0:
+ resolution: {integrity: sha512-vYl3iOX+4CKUWuxGi9Ukhie6fsqXqS9FE2Zaic4tNFD2N2QQaXOMFbuKK4QmDHC0JO6B1Zp41J0LpT0oR68amQ==}
+
range-parser@1.2.0:
resolution: {integrity: sha512-kA5WQoNVo4t9lNx2kQNFCxKeBl5IbbSNBl1M/tLkw9WCn+hxNBAW5Qh8gdhs63CJnhjJ2zQWFoqPJP2sK1AV5A==}
engines: {node: '>= 0.6'}
+ range-parser@1.3.0:
+ resolution: {integrity: sha512-hek2mFQpPuI4E1BBKrSto+BU3e3x4xuarsbiwr3+lf7p44juvFMV0XFWQAP3xUyqXA4RrXLIoaSUGbSt056ZMw==}
+ engines: {node: '>= 0.6'}
+
rc@1.2.8:
resolution: {integrity: sha512-y3bGgqKj3QBdxLbLkomlohkvsA8gdAiUQlSBJnBhfn+BPxg4bc62d8TcBW15wavDfgexCgccckhcZvywyQYPOw==}
hasBin: true
@@ -4058,6 +4485,9 @@ packages:
react: ^16.0.0 || ^17.0.0 || ^18.0.0 || ^19.0.0
react-dom: ^16.0.0 || ^17.0.0 || ^18.0.0 || ^19.0.0
+ record-cache@1.2.0:
+ resolution: {integrity: sha512-kyy3HWCez2WrotaL3O4fTn0rsIdfRKOdQQcEJ9KpvmKmbffKVvwsloX063EgRUlpJIXHiDQFhJcTbZequ2uTZw==}
+
reflect.getprototypeof@1.0.10:
resolution: {integrity: sha512-00o4I+DVrefhv+nX0ulyi3biSHCPDe+yLv5o/p6d/UVlirijB8E16FtfwSAi4g3tcqrQ4lRAqQSoFEZJehYEcw==}
engines: {node: '>= 0.4'}
@@ -4073,6 +4503,10 @@ packages:
resolution: {integrity: sha512-ZbgR5aZEdf4UKZVBPYIgaglBmSF2Hi94s2PcIHhRGFjKYu+chjJdYfHn4rt3hB6eCKLJ8giVIIfgMa1ehDfZKA==}
engines: {node: '>=0.10.0'}
+ require-addon@1.3.0:
+ resolution: {integrity: sha512-j8pifaPNZI7ShKY3ihXZwF0qaEjua8zmlqPvDil0a8SBnyRvjCbAiEQuzO0UcoXH1sqU5U/ilXSjYd1W5NYwAA==}
+ engines: {bare: '>=1.10.0'}
+
require-from-string@2.0.2:
resolution: {integrity: sha512-Xf0nWe6RseziFMu+Ap9biiUbmplq6S9/p+7w7YXP/JBHhrUDDUhwa+vANyubuqfZWTveU//DYVGsDG7RKL/vEw==}
engines: {node: '>=0.10.0'}
@@ -4093,9 +4527,15 @@ packages:
resolution: {integrity: sha512-g6QUff04oZpHs0eG5p83rFLhHeV00ug/Yf9nZM6fLeUrPguBTkTQOdpAWWspMh55TZfVQDPaN3NQJfbVRAxdIw==}
engines: {iojs: '>=1.0.0', node: '>=0.10.0'}
+ run-parallel-limit@1.1.0:
+ resolution: {integrity: sha512-jJA7irRNM91jaKc3Hcl1npHsFLOXOoTkPCUL1JEa1R82O2miplXXRaGdjW/KM/98YQWDhJLiSs793CnXfblJUw==}
+
run-parallel@1.2.0:
resolution: {integrity: sha512-5l4VyZR86LZ/lDxZTR6jqL8AFE2S0IFLMP26AbjsLVADxHdhB/c0GUsH+y39UfCi3dzz8OlQuPmnaJOMoDHQBA==}
+ run-series@1.1.9:
+ resolution: {integrity: sha512-Arc4hUN896vjkqCYrUXquBFtRZdv1PfLbTYP71efP6butxyQ0kWpiNJyAgsxscmQg1cqvHY32/UCBzXedTpU2g==}
+
safe-array-concat@1.1.4:
resolution: {integrity: sha512-wtZlHyOje6OZTGqAoaDKxFkgRtkF9CnHAVnCHKfuj200wAgL+bSJhdsCD2l0Qx/2ekEXjPWcyKkfGb5CPboslg==}
engines: {node: '>=0.4'}
@@ -4111,6 +4551,10 @@ packages:
resolution: {integrity: sha512-x/+Cz4YrimQxQccJf5mKEbIa1NzeCRNI5Ecl/ekmlYaampdNLPalVyIcCZNNH3MvmqBugV5TMYZXv0ljslUlaw==}
engines: {node: '>= 0.4'}
+ sax@1.6.1:
+ resolution: {integrity: sha512-42tBVwLWnaQvW5zc4HbZrTuWccECCZfBi92FDuwtqxasH+JbPB3/FOKb1m222K42R4WxuxzzMsTswfzgtSu64Q==}
+ engines: {node: '>=11.0.0'}
+
scheduler@0.27.0:
resolution: {integrity: sha512-eNv+WrVbKu1f3vbYJT/xtiF5syA5HPIMtf9IgY/nKg0sWqzAUEvqY/xm7OcZc/qafLx/iO9FgOmeSAp4v5ti/Q==}
@@ -4192,6 +4636,20 @@ packages:
resolution: {integrity: sha512-bzyZ1e88w9O1iNJbKnOlvYTrWPDl46O1bG0D3XInv+9tkPrxrN8jUUTiFlDkkmKWgn1M6CfIA13SuGqOa9Korw==}
engines: {node: '>=14'}
+ simple-concat@1.0.1:
+ resolution: {integrity: sha512-cSFtAPtRhljv69IK0hTVZQ+OfE9nePi/rtJmw5UjHeVyVroEqJXP1sFztKUy1qU+xvz3u/sfYJLa947b7nAN2Q==}
+
+ simple-get@4.0.1:
+ resolution: {integrity: sha512-brv7p5WgH0jmQJr1ZDDfKDOSeWWg+OVypG99A/5vYGPqJ6pxiaHLy8nxtFjBA7oMa01ebA9gfh1uMCFqOuXxvA==}
+
+ smart-buffer@4.2.0:
+ resolution: {integrity: sha512-94hK0Hh8rPqQl2xXc3HsaBoOXKV20MToPkcXvwbISWLEs+64sBq5kFgn2kJDHb1Pry9yrP0dxrCI9RRci7RXKg==}
+ engines: {node: '>= 6.0.0', npm: '>= 3.0.0'}
+
+ socks@2.8.10:
+ resolution: {integrity: sha512-e0VyvkVTwVYViNovRkZ9aodhxVlyoMn7eJhVUPxZ+eK9P/7CBkxvvsBOHqFPEH416726W8tLXXXjKwqgTErrCQ==}
+ engines: {node: '>= 10.0.0', npm: '>= 3.0.0'}
+
sonner@2.0.7:
resolution: {integrity: sha512-W6ZN4p58k8aDKA4XPcx2hpIQXBRAgyiWVkYhT7CvK6D3iAu7xjvVyhQHg2/iaKJZ1XVJ4r7XuwGL+WGEK37i9w==}
peerDependencies:
@@ -4202,6 +4660,12 @@ packages:
resolution: {integrity: sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==}
engines: {node: '>=0.10.0'}
+ speed-limiter@1.0.2:
+ resolution: {integrity: sha512-Ax+TbUOho84bWUc3AKqWtkIvAIVws7d6QI4oJkgH4yQ5Yil+lR3vjd/7qd51dHKGzS5bFxg0++QwyNRN7s6rZA==}
+
+ split@1.0.1:
+ resolution: {integrity: sha512-mTyOoPbrivtXnwnIxZRFYRrPNtEFKlpB2fvjSnCQUiAA6qAZzqwna5envK4uk6OIeP17CsdF3rSBGYVBsU0Tkg==}
+
stable-hash@0.0.5:
resolution: {integrity: sha512-+L3ccpzibovGXFK+Ap/f8LOS0ahMrHTf3xu7mMLSpEGU0EO9ucaysSylKo9eRDFNhWve/y275iPmIZ4z39a9iA==}
@@ -4212,6 +4676,12 @@ packages:
stream-browserify@3.0.0:
resolution: {integrity: sha512-H73RAHsVBapbim0tU2JwwOiXUj+fikfiaoYAKHF3VJfA0pe2BCzkhAHBlLG6REzE+2WNZcxOXjK7lkso+9euLA==}
+ streamx@2.28.0:
+ resolution: {integrity: sha512-1Yowhzjf0ivGMrTIkY9hav5TxobO9qIVqUE41fiCGMGgc3CLlf4MY+9AHmZqBWgDTue0fY9zWjYFVyf6Diuobw==}
+
+ streamx@2.28.1:
+ resolution: {integrity: sha512-zEzXb0s5Cds7tqMH6rhZ05lcJydCWiQPEwiNngVqzsxCc962vLY4Uw+mW7od8kDH258k2Uz/JrOkdIAAhSh9VA==}
+
string-width@4.2.3:
resolution: {integrity: sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==}
engines: {node: '>=8'}
@@ -4243,6 +4713,10 @@ packages:
resolution: {integrity: sha512-UXSH262CSZY1tfu3G3Secr6uGLCFVPMhIqHjlgCUtCCcgihYc/xKs9djMTMUOb2j1mVSeU8EU6NWc/iQKU6Gfg==}
engines: {node: '>= 0.4'}
+ string2compact@2.0.2:
+ resolution: {integrity: sha512-ZUUIyrS+sSj84gR6fFoz/boiqCptyw37/hMxIehReuf6ekpJOlwSXhTfEasqc5t3QucBXi2ghwLlHOSC8hjEaw==}
+ engines: {node: '>=12.20.0'}
+
string_decoder@1.3.0:
resolution: {integrity: sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==}
@@ -4262,6 +4736,10 @@ packages:
resolution: {integrity: sha512-BrpvfNAE3dcvq7ll3xVumzjKjZQ5tI1sEUIKr3Uoks0XUl45St3FlatVqef9prk4jRDzhW6WZg+3bk93y6pLjA==}
engines: {node: '>=6'}
+ strip-final-newline@3.0.0:
+ resolution: {integrity: sha512-dOESqjYr96iWYylGObzd39EuNTa5VJxyvVAEm5Jnh7KGo75V43Hk1odPQkNDyXNmUR6k+gEiDVXnjB8HJ3crXw==}
+ engines: {node: '>=12'}
+
strip-final-newline@4.0.0:
resolution: {integrity: sha512-aulFJcD6YK8V1G7iRB5tigAP4TsHBZZrOV8pjV++zdUwmeV8uzbY7yn6h9MswN62adStNZFuCIx4haBnRuMDaw==}
engines: {node: '>=18'}
@@ -4309,6 +4787,31 @@ packages:
resolution: {integrity: sha512-uxc/zpqFg6x7C8vOE7lh6Lbda8eEL9zmVm/PLeTPBRhh1xCgdWaQ+J1CUieGpIfm2HdtsUpRv+HshiasBMcc6A==}
engines: {node: '>=6'}
+ tar-fs@2.1.5:
+ resolution: {integrity: sha512-OboTd8mmMhZDNPV+UjQcK9yKAatXu2aJ+r1w4im1Otd4M4fl2hwvdoXUxIYHFTHWK/3y3FarBP70v3vwmGlOxw==}
+
+ tar-stream@2.2.0:
+ resolution: {integrity: sha512-ujeqbceABgwMZxEJnk2HDY2DlnUZ+9oEcb1KzTVfYHio0UE6dG71n60d8D2I4qNvleWrrXpmjpt7vZeF1LnMZQ==}
+ engines: {node: '>=6'}
+
+ teex@1.0.1:
+ resolution: {integrity: sha512-eYE6iEI62Ni1H8oIa7KlDU6uQBtqr4Eajni3wX7rpfXD8ysFx8z0+dri+KWEPWpBsxXfxu58x/0jvTVT1ekOSg==}
+
+ text-decoder@1.2.7:
+ resolution: {integrity: sha512-vlLytXkeP4xvEq2otHeJfSQIRyWxo/oZGEbXrtEEF9Hnmrdly59sUbzZ/QgyWuLYHctCHxFF4tRQZNQ9k60ExQ==}
+
+ through@2.3.8:
+ resolution: {integrity: sha512-w89qg7PI8wAdvX60bMDP+bFoD5Dvhm9oLheFp5O4a2QF0cSBGsBX4qZmadPMvVqlLJBBci+WqGGOAPvcDeNSVg==}
+
+ throughput@1.0.2:
+ resolution: {integrity: sha512-jvK1ZXuhsggjb3qYQjMiU/AVYYiTeqT5thWvYR2yuy2LGM84P5MSSyAinwHahGsdBYKR9m9HncVR/3f3nFKkxg==}
+
+ thunky@1.1.0:
+ resolution: {integrity: sha512-eHY7nBftgThBqOyHGVN+l8gF0BucP09fMo0oO/Lb0w1OF80dJv+lDVpXG60WMQvkcxAkNybKsrEIE3ZtKGmPrA==}
+
+ timeout-refresh@1.0.3:
+ resolution: {integrity: sha512-Mz0CX4vBGM5lj8ttbIFt7o4ZMxk/9rgudJRh76EvB7xXZMur7T/cjRiH2w4Fmkq0zxf2QpM8IFvOSRn8FEu3gA==}
+
tiny-invariant@1.3.3:
resolution: {integrity: sha512-+FbBPE1o9QAYvviau/qC5SE3caw21q3xkvWKBtja5vgqOWIHHJ3ioaq1VPfn/Szqctz2bU/oYeKd9/z5BL+PVg==}
@@ -4320,6 +4823,14 @@ packages:
resolution: {integrity: sha512-65P7iz6X5yEr1cwcgvQxbbIw7Uk3gOy5dIdtZ4rDveLqhrdJP+Li/Hx6tyK0NEb+2GCyneCMJiGqrADCSNk8sQ==}
engines: {node: '>=8.0'}
+ torrent-discovery@11.0.21:
+ resolution: {integrity: sha512-x+RGu60qJp2u4S7B7XglyYCrfDJWKB0RIexODlA5dpBpMh94q4MbmRjusdbvxN+XD4NHhpwEfub/CRA9pUXjEQ==}
+ engines: {node: '>=16.0.0'}
+
+ torrent-piece@4.0.1:
+ resolution: {integrity: sha512-bL5lLm0CknuXMJ3c8Xw+27etPTH7P1vLXwEASQ3bktZlWV2ZEkCW1l0ssV6Y9EHpcUlf0aqB9x49bkgMA4JrHw==}
+ engines: {node: '>=12.20.0'}
+
ts-api-utils@2.5.0:
resolution: {integrity: sha512-OJ/ibxhPlqrMM0UiNHJ/0CKQkoKF243/AEmplt3qpRgkW8VG7IfOS41h7V8TjITqdByHzrjcS/2si+y4lIh8NA==}
engines: {node: '>=18.12'}
@@ -4337,6 +4848,9 @@ packages:
engines: {node: '>=18.0.0'}
hasBin: true
+ tunnel-agent@0.6.0:
+ resolution: {integrity: sha512-McnNiV1l8RYeY8tBgEpuodCC1mLUdbSN+CYBL7kJsJNInOP8UjDDEwdk6Mw60vdLLrr5NHKZhMAOSrR2NZuQ+w==}
+
tw-animate-css@1.4.0:
resolution: {integrity: sha512-7bziOlRqH0hJx80h/3mbicLW7o8qLsH5+RaLR2t+OHM3D0JlWGODQKQ4cxbK7WlvmUxpcj6Kgu6EKqjrGFe3QQ==}
@@ -4376,6 +4890,9 @@ packages:
engines: {node: '>=14.17'}
hasBin: true
+ uint8-util@2.3.2:
+ resolution: {integrity: sha512-u1c0/eiT1IW+c5+ogbmE2Fu2hmtJQPkuzDmXD/4crMU4FPMjkTkFnjJ2NN/9AsZ9YWcc7yE869uc3WB8wIZEEA==}
+
unbox-primitive@1.1.0:
resolution: {integrity: sha512-nWJ91DjeOkej/TA8pXQ3myruKpKEYgqvpw9lz4OPHj/NWFNluYrjbz9j01CJ8yKQd2g4jFoOkINCTW2I5LEEyw==}
engines: {node: '>= 0.4'}
@@ -4398,6 +4915,12 @@ packages:
resolution: {integrity: sha512-gptHNQghINnc/vTGIk0SOFGFNXw7JVrlRUtConJRlvaw6DuX0wO5Jeko9sWrMBhh+PsYAZ7oXAiOnf/UKogyiw==}
engines: {node: '>= 10.0.0'}
+ unordered-array-remove@1.0.2:
+ resolution: {integrity: sha512-45YsfD6svkgaCBNyvD+dFHm4qFX9g3wRSIVgWVPtm2OCnphvPxzJoe20ATsiNpNJrmzHifnxm+BN5F7gFT/4gw==}
+
+ unordered-set@2.0.1:
+ resolution: {integrity: sha512-eUmNTPzdx+q/WvOHW0bgGYLWvWHNT3PTKEQLg0MAQhc0AHASHVHoP/9YytYd4RBVariqno/mEUhVZN98CmD7bg==}
+
unrs-resolver@1.11.1:
resolution: {integrity: sha512-bSjt9pjaEBnNiGgc9rUiHGKv5l4/TGzDmYw3RhnkJGtLhbnnA/5qJj7x3dNDCRx/PJxu774LlH8lCOlB4hEfKg==}
@@ -4433,9 +4956,26 @@ packages:
'@types/react':
optional: true
+ ut_metadata@5.0.1:
+ resolution: {integrity: sha512-V+6jKXXvkH8YQDFeSpvjSkn7Yiz7Juz3LhIkA1QhxuvOjhrqwlZKJfRMOjVSEW/mPa9Bpk4hvFOCim98tzR04w==}
+ engines: {node: '>=22'}
+
+ ut_pex@5.0.2:
+ resolution: {integrity: sha512-tPWnEmFIpn47IgZ3CyMzA4onr6zGtBrR+DVtRGoyCpC59OgqELvj9jkzLuisCwIH5FdUQiwqtXtsiAWwpUfrKw==}
+ engines: {node: '>=12.20.0'}
+
+ utf-8-validate@6.0.6:
+ resolution: {integrity: sha512-q3l3P9UtEEiAHcsgsqTgf9PPjctrDWoIXW3NpOHFdRDbLvu4DLIcxHangJ4RLrWkBcKjmcs/6NkerI8T/rE4LA==}
+ engines: {node: '>=6.14.2'}
+
util-deprecate@1.0.2:
resolution: {integrity: sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==}
+ utp-native@2.5.3:
+ resolution: {integrity: sha512-sWTrWYXPhhWJh+cS2baPzhaZc89zwlWCfwSthUjGhLkZztyPhcQllo+XVVCbNGi7dhyRlxkWxN4NKU6FbA9Y8w==}
+ engines: {node: '>=8.12'}
+ hasBin: true
+
vary@1.1.2:
resolution: {integrity: sha512-BNGbWLfd0eUPabhkXUVm0j8uuvREyTh5ovRa/dyow/BqAbZJyC+5fU+IzQOzmAKzYqYRAISoRhdQr3eIZ/PXqg==}
engines: {node: '>= 0.8'}
@@ -4446,6 +4986,18 @@ packages:
weak-lru-cache@1.2.2:
resolution: {integrity: sha512-DEAoo25RfSYMuTGc9vPJzZcZullwIqRDSI9LOy+fkCJPi6hykCnfKaXTuPBDuXAUcqHXyOgFtHNp/kB2FjYHbw==}
+ web-streams-polyfill@3.3.3:
+ resolution: {integrity: sha512-d2JWLCivmZYTSIoge9MsgFCZrt571BikcWGYkjC1khllbTeDlGqZ2D8vD8E/lJa8WGWbb7Plm8/XJYV7IJHZZw==}
+ engines: {node: '>= 8'}
+
+ webrtc-polyfill@1.2.2:
+ resolution: {integrity: sha512-t/H2OySa7lZSq2f5j4whmr+IWCl5TqF4daGotE8VhPP7U+E4x5x8MK1wqOkqlb4xxRlag4kBB9enU4oxNk//+g==}
+ engines: {node: '>=16.0.0'}
+
+ webtorrent@3.0.21:
+ resolution: {integrity: sha512-PFgLphma0dsUWmbWrZ016Cja+j+3D3DXuhk09A5u9qVAzH1T4Vj1VZd8j+zb8sDwPv7NA7+qgPtDEN4iFd6wdw==}
+ engines: {node: '>=22'}
+
which-boxed-primitive@1.1.1:
resolution: {integrity: sha512-TbX3mj8n0odCBFVlY8AxkqcHASw3L60jIuF8jFP78az3C2YhmGvqbHBpAjTRH2/xqYunrJ9g1jSyjCjpoWzIAA==}
engines: {node: '>= 0.4'}
@@ -4458,6 +5010,9 @@ packages:
resolution: {integrity: sha512-K4jVyjnBdgvc86Y6BkaLZEN933SwYOuBFkdmBu9ZfkcAbdVbpITnDmjvZ/aQjRXQrv5EPkTnD1s39GiiqbngCw==}
engines: {node: '>= 0.4'}
+ which-runtime@1.4.0:
+ resolution: {integrity: sha512-0ugbP4CJW4e2D20jvEcC4973dCgIaHI4Rw1PT+26U9zEve7FyYdWAIwUnoeOYvoCfn+wXHoHTKb1KhkYlb60Pw==}
+
which-typed-array@1.1.20:
resolution: {integrity: sha512-LYfpUkmqwl0h9A2HL09Mms427Q1RZWuOHsukfVcKRq9q95iQxdw0ix1JQrqbcDR9PH1QDwf5Qo8OZb5lksZ8Xg==}
engines: {node: '>= 0.4'}
@@ -4494,6 +5049,9 @@ packages:
resolution: {integrity: sha512-si7QWI6zUMq56bESFvagtmzMdGOtoxfR+Sez11Mobfc7tm+VkUckk9bW2UeffTGVUbOksxmSw0AA2gs8g71NCQ==}
engines: {node: '>=12'}
+ wrappy@1.0.2:
+ resolution: {integrity: sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==}
+
ws@8.21.0:
resolution: {integrity: sha512-Vsp28b7DRcimFQvrqu2Wek3z1iYxDCWqHYB8Qsnk/S4RfaCQzPGPyBNuVjJV3cd6UiKtUtp6sNM77gWvzcCH+g==}
engines: {node: '>=10.0.0'}
@@ -4506,6 +5064,26 @@ packages:
utf-8-validate:
optional: true
+ ws@8.22.0:
+ resolution: {integrity: sha512-Ydggc987+RO0AnWtZ/7Wq9FtNvcrL1b/RO0ud9mWjUPgDrsAAwQSF51sm2hm1XofbU/4jkpGEsLFsZZxU+1DOg==}
+ engines: {node: '>=10.0.0'}
+ peerDependencies:
+ bufferutil: ^4.0.1
+ utf-8-validate: '>=5.0.2'
+ peerDependenciesMeta:
+ bufferutil:
+ optional: true
+ utf-8-validate:
+ optional: true
+
+ xml2js@0.6.2:
+ resolution: {integrity: sha512-T4rieHaC1EXcES0Kxxj4JWgaUQHDk+qwHcYOCFHfiwKz7tOVPLq7Hjq9dM1WCMhylqMEfP7hMcOIChvotiZegA==}
+ engines: {node: '>=4.0.0'}
+
+ xmlbuilder@11.0.1:
+ resolution: {integrity: sha512-fDlsI/kFEx7gLvbecc0/ohLG50fugQp8ryHzMTuW9vSa1GJ0XYWKnhsUx7oie3G98+r56aTQIUB4kht42R3JvA==}
+ engines: {node: '>=4.0'}
+
yallist@3.1.1:
resolution: {integrity: sha512-a4UGQaWPH59mOXUYnAG2ewncQS4i4F43Tv3JoAM+s2VDAmS9NsK8GpDMLrCHPksFT7h3K6TOoUNn2pb7RoXx4g==}
@@ -6199,6 +6777,17 @@ snapshots:
'@sec-ant/readable-stream@0.4.1': {}
+ '@silentbot1/nat-api@0.4.9(supports-color@10.2.2)':
+ dependencies:
+ chrome-dgram: 3.0.6
+ cross-fetch-ponyfill: 1.0.3
+ debug: 4.4.3(supports-color@10.2.2)
+ default-gateway: 7.2.2
+ unordered-array-remove: 1.0.2
+ xml2js: 0.6.2
+ transitivePeerDependencies:
+ - supports-color
+
'@sindresorhus/is@7.2.0': {}
'@sindresorhus/merge-streams@4.0.0': {}
@@ -6333,6 +6922,36 @@ snapshots:
'@tanstack/virtual-core@3.14.0': {}
+ '@thaunknown/simple-peer@10.1.2(supports-color@10.2.2)':
+ dependencies:
+ debug: 4.4.3(supports-color@10.2.2)
+ err-code: 3.0.1
+ streamx: 2.28.1
+ uint8-util: 2.3.2
+ webrtc-polyfill: 1.2.2
+ transitivePeerDependencies:
+ - bare-abort-controller
+ - react-native-b4a
+ - supports-color
+
+ '@thaunknown/simple-websocket@9.1.5(bufferutil@4.1.0)(supports-color@10.2.2)(utf-8-validate@6.0.6)':
+ dependencies:
+ debug: 4.4.3(supports-color@10.2.2)
+ queue-microtask: 1.2.3
+ streamx: 2.28.1
+ uint8-util: 2.3.2
+ ws: 8.22.0(bufferutil@4.1.0)(utf-8-validate@6.0.6)
+ transitivePeerDependencies:
+ - bare-abort-controller
+ - bufferutil
+ - react-native-b4a
+ - supports-color
+ - utf-8-validate
+
+ '@thaunknown/thirty-two@1.0.6':
+ dependencies:
+ uint8-util: 2.3.2
+
'@tybys/wasm-util@0.10.1':
dependencies:
tslib: 2.8.1
@@ -6457,7 +7076,7 @@ snapshots:
'@typescript-eslint/visitor-keys': 8.59.0
debug: 4.4.3(supports-color@10.2.2)
minimatch: 10.2.5
- semver: 7.7.4
+ semver: 7.8.5
tinyglobby: 0.2.16
ts-api-utils: 2.5.0(typescript@5.9.3)
typescript: 5.9.3
@@ -6539,14 +7158,25 @@ snapshots:
'@unrs/resolver-binding-win32-x64-msvc@1.11.1':
optional: true
+ '@webtorrent/http-node@1.3.0':
+ dependencies:
+ freelist: 1.0.3
+ http-parser-js: 0.4.13
+
'@zeit/schemas@2.36.0': {}
+ abort-controller@3.0.0:
+ dependencies:
+ event-target-shim: 5.0.1
+
acorn-jsx@5.3.2(acorn@8.16.0):
dependencies:
acorn: 8.16.0
acorn@8.16.0: {}
+ addr-to-ip-port@2.0.0: {}
+
ajv@6.15.0:
dependencies:
fast-deep-equal: 3.1.3
@@ -6666,23 +7296,158 @@ snapshots:
axobject-query@4.1.0: {}
+ b4a@1.9.0: {}
+
balanced-match@1.0.2: {}
balanced-match@4.0.4: {}
- base64-js@1.5.1: {}
-
- baseline-browser-mapping@2.10.23: {}
+ bare-addon-resolve@1.11.0(bare-url@2.5.4):
+ dependencies:
+ bare-module-resolve: 1.12.6(bare-url@2.5.4)
+ bare-semver: 1.1.0
+ optionalDependencies:
+ bare-url: 2.5.4
+ optional: true
- blake3-wasm@2.1.5: {}
+ bare-events@2.9.2: {}
- bowser@2.14.1: {}
+ bare-fs@4.8.3:
+ dependencies:
+ bare-events: 2.9.2
+ bare-path: 3.1.2
+ bare-stream: 2.13.4(bare-events@2.9.2)
+ bare-url: 2.5.4
+ fast-fifo: 1.3.2
+ transitivePeerDependencies:
+ - bare-abort-controller
+ - react-native-b4a
- boxen@7.0.0:
+ bare-module-resolve@1.12.6(bare-url@2.5.4):
dependencies:
- ansi-align: 3.0.1
- camelcase: 7.0.1
- chalk: 5.0.1
+ bare-semver: 1.1.0
+ optionalDependencies:
+ bare-url: 2.5.4
+ optional: true
+
+ bare-path@3.1.2: {}
+
+ bare-semver@1.1.0:
+ optional: true
+
+ bare-stream@2.13.4(bare-events@2.9.2):
+ dependencies:
+ b4a: 1.9.0
+ streamx: 2.28.0
+ teex: 1.0.1
+ optionalDependencies:
+ bare-events: 2.9.2
+ transitivePeerDependencies:
+ - react-native-b4a
+
+ bare-url@2.5.4:
+ dependencies:
+ bare-path: 3.1.2
+
+ base64-js@1.5.1: {}
+
+ baseline-browser-mapping@2.10.23: {}
+
+ bencode@2.0.3: {}
+
+ bencode@4.0.1:
+ dependencies:
+ uint8-util: 2.3.2
+
+ bep53-range@2.0.1: {}
+
+ bitfield@5.0.1: {}
+
+ bittorrent-dht@11.0.12(supports-color@10.2.2):
+ dependencies:
+ bencode: 4.0.1
+ debug: 4.4.3(supports-color@10.2.2)
+ k-bucket: 5.1.0
+ k-rpc: 5.1.0
+ last-one-wins: 1.0.4
+ lru: 3.1.0
+ randombytes: 2.1.0
+ record-cache: 1.2.0
+ transitivePeerDependencies:
+ - react-native-b4a
+ - supports-color
+
+ bittorrent-lsd@2.0.3(supports-color@10.2.2):
+ dependencies:
+ chrome-dgram: 3.0.6
+ debug: 4.4.3(supports-color@10.2.2)
+ transitivePeerDependencies:
+ - supports-color
+
+ bittorrent-peerid@2.0.2: {}
+
+ bittorrent-protocol@5.0.9(supports-color@10.2.2):
+ dependencies:
+ bencode: 4.0.1
+ bitfield: 5.0.1
+ debug: 4.4.3(supports-color@10.2.2)
+ streamx: 2.28.0
+ throughput: 1.0.2
+ uint8-util: 2.3.2
+ unordered-array-remove: 1.0.2
+ transitivePeerDependencies:
+ - bare-abort-controller
+ - react-native-b4a
+ - supports-color
+
+ bittorrent-tracker@11.2.3(supports-color@10.2.2):
+ dependencies:
+ '@thaunknown/simple-peer': 10.1.2(supports-color@10.2.2)
+ '@thaunknown/simple-websocket': 9.1.5(bufferutil@4.1.0)(supports-color@10.2.2)(utf-8-validate@6.0.6)
+ bencode: 4.0.1
+ bittorrent-peerid: 2.0.2
+ chrome-dgram: 3.0.6
+ compact2string: 1.4.1
+ cross-fetch-ponyfill: 1.0.3
+ debug: 4.4.3(supports-color@10.2.2)
+ ip: 2.0.1
+ lru: 3.1.0
+ minimist: 1.2.8
+ once: 1.4.0
+ queue-microtask: 1.2.3
+ random-iterate: 1.0.1
+ run-parallel: 1.2.0
+ run-series: 1.1.9
+ socks: 2.8.10
+ string2compact: 2.0.2
+ uint8-util: 2.3.2
+ unordered-array-remove: 1.0.2
+ ws: 8.21.0(bufferutil@4.1.0)(utf-8-validate@6.0.6)
+ optionalDependencies:
+ bufferutil: 4.1.0
+ utf-8-validate: 6.0.6
+ transitivePeerDependencies:
+ - bare-abort-controller
+ - react-native-b4a
+ - supports-color
+
+ bl@4.1.0:
+ dependencies:
+ buffer: 5.6.0
+ inherits: 2.0.4
+ readable-stream: 3.6.2
+
+ blake3-wasm@2.1.5: {}
+
+ block-iterator@1.1.2: {}
+
+ bowser@2.14.1: {}
+
+ boxen@7.0.0:
+ dependencies:
+ ansi-align: 3.0.1
+ camelcase: 7.0.1
+ chalk: 5.0.1
cli-boxes: 3.0.0
string-width: 5.1.2
type-fest: 2.19.0
@@ -6715,10 +7480,20 @@ snapshots:
base64-js: 1.5.1
ieee754: 1.2.1
+ bufferutil@4.1.0:
+ dependencies:
+ node-gyp-build: 4.8.4
+ optional: true
+
bytes@3.0.0: {}
bytes@3.1.2: {}
+ cache-chunk-store@3.2.2:
+ dependencies:
+ lru: 3.1.0
+ queue-microtask: 1.2.3
+
call-bind-apply-helpers@1.0.2:
dependencies:
es-errors: 1.3.0
@@ -6753,6 +7528,25 @@ snapshots:
chalk@5.0.1: {}
+ chownr@1.1.4: {}
+
+ chrome-dgram@3.0.6:
+ dependencies:
+ inherits: 2.0.4
+ run-series: 1.1.9
+
+ chrome-dns@1.0.1:
+ dependencies:
+ chrome-net: 3.3.4
+
+ chrome-net@3.3.4:
+ dependencies:
+ inherits: 2.0.4
+
+ chunk-store-iterator@1.0.4:
+ dependencies:
+ block-iterator: 1.1.2
+
class-variance-authority@0.7.1:
dependencies:
clsx: 2.1.1
@@ -6787,6 +7581,10 @@ snapshots:
color-name@1.1.4: {}
+ compact2string@1.4.1:
+ dependencies:
+ ipaddr.js: 2.5.0
+
compressible@2.0.18:
dependencies:
mime-db: 1.54.0
@@ -6811,6 +7609,28 @@ snapshots:
cookie@1.1.1: {}
+ cpus@1.0.3: {}
+
+ create-torrent@6.1.3:
+ dependencies:
+ bencode: 4.0.1
+ block-iterator: 1.1.2
+ fast-readable-async-iterator: 2.0.0
+ is-file: 1.0.0
+ join-async-iterator: 1.1.1
+ junk: 4.0.1
+ minimist: 1.2.8
+ once: 1.4.0
+ piece-length: 2.0.1
+ queue-microtask: 1.2.3
+ run-parallel: 1.2.0
+ uint8-util: 2.3.2
+
+ cross-fetch-ponyfill@1.0.3:
+ dependencies:
+ abort-controller: 3.0.0
+ node-fetch: 3.3.2
+
cross-spawn@7.0.6:
dependencies:
path-key: 3.1.1
@@ -6859,6 +7679,8 @@ snapshots:
damerau-levenshtein@1.0.8: {}
+ data-uri-to-buffer@4.0.1: {}
+
data-view-buffer@1.0.2:
dependencies:
call-bound: 1.0.4
@@ -6897,12 +7719,20 @@ snapshots:
decimal.js-light@2.5.1: {}
+ decompress-response@6.0.0:
+ dependencies:
+ mimic-response: 3.1.0
+
deep-extend@0.6.0: {}
deep-is@0.1.4: {}
deepmerge@4.3.1: {}
+ default-gateway@7.2.2:
+ dependencies:
+ execa: 7.2.0
+
define-data-property@1.1.4:
dependencies:
es-define-property: 1.0.1
@@ -6942,11 +7772,17 @@ snapshots:
emoji-regex@9.2.2: {}
+ end-of-stream@1.4.5:
+ dependencies:
+ once: 1.4.0
+
enhanced-resolve@5.21.0:
dependencies:
graceful-fs: 4.2.11
tapable: 2.3.3
+ err-code@3.0.1: {}
+
error-stack-parser-es@1.0.5: {}
es-abstract@1.24.2:
@@ -7110,6 +7946,8 @@ snapshots:
escalade@3.2.0: {}
+ escape-html@1.0.3: {}
+
escape-string-regexp@4.0.0: {}
escape-string-regexp@5.0.0: {}
@@ -7319,8 +8157,16 @@ snapshots:
esutils@2.0.3: {}
+ event-target-shim@5.0.1: {}
+
eventemitter3@4.0.7: {}
+ events-universal@1.0.1:
+ dependencies:
+ bare-events: 2.9.2
+ transitivePeerDependencies:
+ - bare-abort-controller
+
events@3.3.0: {}
eventsource-parser@3.1.0: {}
@@ -7341,6 +8187,18 @@ snapshots:
signal-exit: 3.0.7
strip-final-newline: 2.0.0
+ execa@7.2.0:
+ dependencies:
+ cross-spawn: 7.0.6
+ get-stream: 6.0.1
+ human-signals: 4.3.1
+ is-stream: 3.0.0
+ merge-stream: 2.0.0
+ npm-run-path: 5.3.0
+ onetime: 6.0.0
+ signal-exit: 3.0.7
+ strip-final-newline: 3.0.0
+
execa@9.6.1:
dependencies:
'@sindresorhus/merge-streams': 4.0.0
@@ -7356,10 +8214,14 @@ snapshots:
strip-final-newline: 4.0.0
yoctocolors: 2.1.2
+ expand-template@2.0.3: {}
+
fast-deep-equal@3.1.3: {}
fast-equals@5.4.0: {}
+ fast-fifo@1.3.2: {}
+
fast-glob@3.3.1:
dependencies:
'@nodelib/fs.stat': 2.0.5
@@ -7372,6 +8234,8 @@ snapshots:
fast-levenshtein@2.0.6: {}
+ fast-readable-async-iterator@2.0.0: {}
+
fast-uri@3.1.0: {}
fastq@1.20.1:
@@ -7382,6 +8246,11 @@ snapshots:
optionalDependencies:
picomatch: 4.0.4
+ fetch-blob@3.2.0:
+ dependencies:
+ node-domexception: 1.0.0
+ web-streams-polyfill: 3.3.3
+
figures@6.1.0:
dependencies:
is-unicode-supported: 2.1.0
@@ -7390,6 +8259,8 @@ snapshots:
dependencies:
flat-cache: 4.0.1
+ filename-reserved-regex@3.0.0: {}
+
fill-range@7.1.1:
dependencies:
to-regex-range: 5.0.1
@@ -7412,12 +8283,46 @@ snapshots:
dependencies:
is-callable: 1.2.7
+ formdata-polyfill@4.0.10:
+ dependencies:
+ fetch-blob: 3.2.0
+
+ freelist@1.0.3: {}
+
+ fs-chunk-store@5.0.1(bare-url@2.5.4):
+ dependencies:
+ filename-reserved-regex: 3.0.0
+ queue-microtask: 1.2.3
+ random-access-file: 4.1.2(bare-url@2.5.4)
+ run-parallel: 1.2.0
+ thunky: 1.1.0
+ uint8-util: 2.3.2
+ transitivePeerDependencies:
+ - bare-abort-controller
+ - bare-buffer
+ - bare-url
+ - react-native-b4a
+
+ fs-constants@1.0.0: {}
+
fs-extra@11.3.4:
dependencies:
graceful-fs: 4.2.11
jsonfile: 6.2.1
universalify: 2.0.1
+ fs-native-extensions@1.5.1(bare-url@2.5.4):
+ dependencies:
+ require-addon: 1.3.0(bare-url@2.5.4)
+ which-runtime: 1.4.0
+ transitivePeerDependencies:
+ - bare-url
+ optional: true
+
+ fsa-chunk-store@1.3.0:
+ dependencies:
+ filename-reserved-regex: 3.0.0
+
fsevents@2.3.2:
optional: true
@@ -7461,6 +8366,8 @@ snapshots:
dunder-proto: 1.0.1
es-object-atoms: 1.1.1
+ get-stdin@10.0.0: {}
+
get-stream@6.0.1: {}
get-stream@9.0.1:
@@ -7478,6 +8385,8 @@ snapshots:
dependencies:
resolve-pkg-maps: 1.0.0
+ github-from-package@0.0.0: {}
+
glob-parent@5.1.2:
dependencies:
is-glob: 4.0.3
@@ -7529,8 +8438,12 @@ snapshots:
hls.js@1.6.16: {}
+ http-parser-js@0.4.13: {}
+
human-signals@2.1.0: {}
+ human-signals@4.3.1: {}
+
human-signals@8.0.1: {}
ieee754@1.2.1: {}
@@ -7539,6 +8452,10 @@ snapshots:
ignore@7.0.5: {}
+ immediate-chunk-store@2.2.0:
+ dependencies:
+ queue-microtask: 1.2.3
+
import-fresh@3.3.1:
dependencies:
parent-module: 1.0.1
@@ -7558,6 +8475,18 @@ snapshots:
internmap@2.0.3: {}
+ ip-address@10.7.3: {}
+
+ ip-set@3.0.0:
+ dependencies:
+ ip-address: 10.7.3
+
+ ip@2.0.1: {}
+
+ ipaddr.js@2.0.1: {}
+
+ ipaddr.js@2.5.0: {}
+
is-array-buffer@3.0.5:
dependencies:
call-bind: 1.0.9
@@ -7606,6 +8535,8 @@ snapshots:
is-extglob@2.1.1: {}
+ is-file@1.0.0: {}
+
is-finalizationregistry@1.1.1:
dependencies:
call-bound: 1.0.4
@@ -7654,6 +8585,8 @@ snapshots:
is-stream@2.0.1: {}
+ is-stream@3.0.0: {}
+
is-stream@4.0.1: {}
is-string@1.1.1:
@@ -7703,6 +8636,8 @@ snapshots:
jiti@2.6.1: {}
+ join-async-iterator@1.1.1: {}
+
jose@6.2.3: {}
js-tokens@4.0.0: {}
@@ -7740,6 +8675,25 @@ snapshots:
object.assign: 4.1.7
object.values: 1.2.1
+ junk@4.0.1: {}
+
+ k-bucket@5.1.0:
+ dependencies:
+ randombytes: 2.1.0
+
+ k-rpc-socket@1.11.1:
+ dependencies:
+ bencode: 2.0.3
+ chrome-dgram: 3.0.6
+ chrome-dns: 1.0.1
+ chrome-net: 3.3.4
+
+ k-rpc@5.1.0:
+ dependencies:
+ k-bucket: 5.1.0
+ k-rpc-socket: 1.11.1
+ randombytes: 2.1.0
+
keyv@4.5.4:
dependencies:
json-buffer: 3.0.1
@@ -7752,6 +8706,8 @@ snapshots:
dependencies:
language-subtag-registry: 0.3.23
+ last-one-wins@1.0.4: {}
+
levn@0.4.1:
dependencies:
prelude-ls: 1.2.1
@@ -7806,6 +8762,8 @@ snapshots:
lightningcss-win32-arm64-msvc: 1.32.0
lightningcss-win32-x64-msvc: 1.32.0
+ limiter@1.1.5: {}
+
lmdb@3.5.4:
dependencies:
'@harperfast/extended-iterable': 1.0.3
@@ -7823,6 +8781,15 @@ snapshots:
'@lmdb/lmdb-win32-arm64': 3.5.4
'@lmdb/lmdb-win32-x64': 3.5.4
+ load-ip-set@3.0.2:
+ dependencies:
+ cross-fetch-ponyfill: 1.0.3
+ ip-set: 3.0.0
+ netmask: 2.1.1
+ once: 1.4.0
+ queue-microtask: 1.2.3
+ split: 1.0.1
+
load-script@1.0.0: {}
locate-path@6.0.0:
@@ -7841,6 +8808,17 @@ snapshots:
dependencies:
yallist: 3.1.1
+ lru@3.1.0:
+ dependencies:
+ inherits: 2.0.4
+
+ lt_donthave@2.0.7(supports-color@10.2.2):
+ dependencies:
+ debug: 4.4.3(supports-color@10.2.2)
+ unordered-array-remove: 1.0.2
+ transitivePeerDependencies:
+ - supports-color
+
lucide-react@1.16.0(react@19.2.4):
dependencies:
react: 19.2.4
@@ -7849,6 +8827,12 @@ snapshots:
dependencies:
'@jridgewell/sourcemap-codec': 1.5.5
+ magnet-uri@7.0.10:
+ dependencies:
+ '@thaunknown/thirty-two': 1.0.6
+ bep53-range: 2.0.1
+ uint8-util: 2.3.2
+
markdown-to-jsx@7.7.17(react@19.2.4):
optionalDependencies:
react: 19.2.4
@@ -7857,6 +8841,10 @@ snapshots:
memoize-one@5.2.1: {}
+ memory-chunk-store@1.3.5:
+ dependencies:
+ queue-microtask: 1.2.3
+
merge-stream@2.0.0: {}
merge2@1.4.1: {}
@@ -7874,15 +8862,21 @@ snapshots:
dependencies:
mime-db: 1.33.0
+ mime@3.0.0: {}
+
mimic-fn@2.1.0: {}
- miniflare@5.20261001.0-alpha(@types/node@20.19.39):
+ mimic-fn@4.0.0: {}
+
+ mimic-response@3.1.0: {}
+
+ miniflare@5.20261001.0-alpha(@types/node@20.19.39)(bufferutil@4.1.0)(utf-8-validate@6.0.6):
dependencies:
'@cspotcode/source-map-support': 0.8.1
sharp: 0.35.4(@types/node@20.19.39)
undici: 7.29.1
workerd: 1.20261001.1
- ws: 8.21.0
+ ws: 8.21.0(bufferutil@4.1.0)(utf-8-validate@6.0.6)
youch: 4.1.0-beta.10
transitivePeerDependencies:
- '@types/node'
@@ -7899,6 +8893,8 @@ snapshots:
minimist@1.2.8: {}
+ mkdirp-classic@0.5.3: {}
+
ms@2.0.0: {}
ms@2.1.3: {}
@@ -7921,12 +8917,19 @@ snapshots:
nanoid@3.3.11: {}
+ napi-build-utils@2.0.0: {}
+
+ napi-macros@2.2.2:
+ optional: true
+
napi-postinstall@0.3.4: {}
natural-compare@1.4.0: {}
negotiator@0.6.4: {}
+ netmask@2.1.1: {}
+
next@16.2.3(@babel/core@7.29.0(supports-color@10.2.2))(@playwright/test@1.59.1)(react-dom@19.2.4(react@19.2.4))(react@19.2.4):
dependencies:
'@next/env': 16.2.3
@@ -7952,8 +8955,18 @@ snapshots:
- '@babel/core'
- babel-plugin-macros
+ node-abi@3.98.0:
+ dependencies:
+ semver: 7.8.5
+
node-addon-api@6.1.0: {}
+ node-datachannel@0.32.3:
+ dependencies:
+ prebuild-install: 7.1.3
+
+ node-domexception@1.0.0: {}
+
node-exports-info@1.6.0:
dependencies:
array.prototype.flatmap: 1.3.3
@@ -7961,16 +8974,29 @@ snapshots:
object.entries: 1.1.9
semver: 6.3.1
+ node-fetch@3.3.2:
+ dependencies:
+ data-uri-to-buffer: 4.0.1
+ fetch-blob: 3.2.0
+ formdata-polyfill: 4.0.10
+
node-gyp-build-optional-packages@5.2.2:
dependencies:
detect-libc: 2.1.2
+ node-gyp-build@4.8.4:
+ optional: true
+
node-releases@2.0.38: {}
npm-run-path@4.0.1:
dependencies:
path-key: 3.1.1
+ npm-run-path@5.3.0:
+ dependencies:
+ path-key: 4.0.0
+
npm-run-path@6.0.0:
dependencies:
path-key: 4.0.0
@@ -8020,10 +9046,18 @@ snapshots:
on-headers@1.1.0: {}
+ once@1.4.0:
+ dependencies:
+ wrappy: 1.0.2
+
onetime@5.1.2:
dependencies:
mimic-fn: 2.1.0
+ onetime@6.0.0:
+ dependencies:
+ mimic-fn: 4.0.0
+
optionator@0.9.4:
dependencies:
deep-is: 0.1.4
@@ -8059,6 +9093,15 @@ snapshots:
parse-ms@4.0.0: {}
+ parse-torrent@11.0.24:
+ dependencies:
+ bencode: 4.0.1
+ cross-fetch-ponyfill: 1.0.3
+ get-stdin: 10.0.0
+ magnet-uri: 7.0.10
+ queue-microtask: 1.2.3
+ uint8-util: 2.3.2
+
path-exists@4.0.0: {}
path-is-inside@1.0.2: {}
@@ -8081,6 +9124,8 @@ snapshots:
picomatch@4.0.4: {}
+ piece-length@2.0.1: {}
+
pkce-challenge@5.0.1: {}
playwright-core@1.59.1: {}
@@ -8105,6 +9150,21 @@ snapshots:
picocolors: 1.1.1
source-map-js: 1.2.1
+ prebuild-install@7.1.3:
+ dependencies:
+ detect-libc: 2.1.2
+ expand-template: 2.0.3
+ github-from-package: 0.0.0
+ minimist: 1.2.8
+ mkdirp-classic: 0.5.3
+ napi-build-utils: 2.0.0
+ node-abi: 3.98.0
+ pump: 3.0.4
+ rc: 1.2.8
+ simple-get: 4.0.1
+ tar-fs: 2.1.5
+ tunnel-agent: 0.6.0
+
prelude-ls@1.2.1: {}
pretty-ms@9.3.0:
@@ -8117,10 +9177,17 @@ snapshots:
object-assign: 4.1.1
react-is: 16.13.1
+ pump@3.0.4:
+ dependencies:
+ end-of-stream: 1.4.5
+ once: 1.4.0
+
punycode@2.3.1: {}
queue-microtask@1.2.3: {}
+ queue-tick@1.0.1: {}
+
radix-ui@1.6.0(@types/react-dom@19.2.3(@types/react@19.2.14))(@types/react@19.2.14)(react-dom@19.2.4(react@19.2.4))(react@19.2.4):
dependencies:
'@radix-ui/primitive': 1.1.4
@@ -8184,8 +9251,36 @@ snapshots:
'@types/react': 19.2.14
'@types/react-dom': 19.2.3(@types/react@19.2.14)
+ random-access-file@4.1.2(bare-url@2.5.4):
+ dependencies:
+ bare-fs: 4.8.3
+ bare-path: 3.1.2
+ random-access-storage: 3.0.2
+ optionalDependencies:
+ fs-native-extensions: 1.5.1(bare-url@2.5.4)
+ transitivePeerDependencies:
+ - bare-abort-controller
+ - bare-buffer
+ - bare-url
+ - react-native-b4a
+
+ random-access-storage@3.0.2:
+ dependencies:
+ bare-events: 2.9.2
+ queue-tick: 1.0.1
+ transitivePeerDependencies:
+ - bare-abort-controller
+
+ random-iterate@1.0.1: {}
+
+ randombytes@2.1.0:
+ dependencies:
+ safe-buffer: 5.2.1
+
range-parser@1.2.0: {}
+ range-parser@1.3.0: {}
+
rc@1.2.8:
dependencies:
deep-extend: 0.6.0
@@ -8282,6 +9377,12 @@ snapshots:
tiny-invariant: 1.3.3
victory-vendor: 36.9.2
+ record-cache@1.2.0:
+ dependencies:
+ b4a: 1.9.0
+ transitivePeerDependencies:
+ - react-native-b4a
+
reflect.getprototypeof@1.0.10:
dependencies:
call-bind: 1.0.9
@@ -8311,6 +9412,13 @@ snapshots:
dependencies:
rc: 1.2.8
+ require-addon@1.3.0(bare-url@2.5.4):
+ dependencies:
+ bare-addon-resolve: 1.11.0(bare-url@2.5.4)
+ transitivePeerDependencies:
+ - bare-url
+ optional: true
+
require-from-string@2.0.2: {}
resolve-from@4.0.0: {}
@@ -8328,10 +9436,16 @@ snapshots:
reusify@1.1.0: {}
+ run-parallel-limit@1.1.0:
+ dependencies:
+ queue-microtask: 1.2.3
+
run-parallel@1.2.0:
dependencies:
queue-microtask: 1.2.3
+ run-series@1.1.9: {}
+
safe-array-concat@1.1.4:
dependencies:
call-bind: 1.0.9
@@ -8353,6 +9467,8 @@ snapshots:
es-errors: 1.3.0
is-regex: 1.2.1
+ sax@1.6.1: {}
+
scheduler@0.27.0: {}
semver@6.3.1: {}
@@ -8512,6 +9628,21 @@ snapshots:
signal-exit@4.1.0: {}
+ simple-concat@1.0.1: {}
+
+ simple-get@4.0.1:
+ dependencies:
+ decompress-response: 6.0.0
+ once: 1.4.0
+ simple-concat: 1.0.1
+
+ smart-buffer@4.2.0: {}
+
+ socks@2.8.10:
+ dependencies:
+ ip-address: 10.7.3
+ smart-buffer: 4.2.0
+
sonner@2.0.7(react-dom@19.2.4(react@19.2.4))(react@19.2.4):
dependencies:
react: 19.2.4
@@ -8519,6 +9650,18 @@ snapshots:
source-map-js@1.2.1: {}
+ speed-limiter@1.0.2:
+ dependencies:
+ limiter: 1.1.5
+ streamx: 2.28.0
+ transitivePeerDependencies:
+ - bare-abort-controller
+ - react-native-b4a
+
+ split@1.0.1:
+ dependencies:
+ through: 2.3.8
+
stable-hash@0.0.5: {}
stop-iteration-iterator@1.1.0:
@@ -8531,6 +9674,24 @@ snapshots:
inherits: 2.0.4
readable-stream: 3.6.2
+ streamx@2.28.0:
+ dependencies:
+ events-universal: 1.0.1
+ fast-fifo: 1.3.2
+ text-decoder: 1.2.7
+ transitivePeerDependencies:
+ - bare-abort-controller
+ - react-native-b4a
+
+ streamx@2.28.1:
+ dependencies:
+ events-universal: 1.0.1
+ fast-fifo: 1.3.2
+ text-decoder: 1.2.7
+ transitivePeerDependencies:
+ - bare-abort-controller
+ - react-native-b4a
+
string-width@4.2.3:
dependencies:
emoji-regex: 8.0.0
@@ -8593,6 +9754,11 @@ snapshots:
define-properties: 1.2.1
es-object-atoms: 1.1.1
+ string2compact@2.0.2:
+ dependencies:
+ addr-to-ip-port: 2.0.0
+ ipaddr.js: 2.0.1
+
string_decoder@1.3.0:
dependencies:
safe-buffer: 5.2.1
@@ -8609,6 +9775,8 @@ snapshots:
strip-final-newline@2.0.0: {}
+ strip-final-newline@3.0.0: {}
+
strip-final-newline@4.0.0: {}
strip-json-comments@2.0.1: {}
@@ -8636,6 +9804,43 @@ snapshots:
tapable@2.3.3: {}
+ tar-fs@2.1.5:
+ dependencies:
+ chownr: 1.1.4
+ mkdirp-classic: 0.5.3
+ pump: 3.0.4
+ tar-stream: 2.2.0
+
+ tar-stream@2.2.0:
+ dependencies:
+ bl: 4.1.0
+ end-of-stream: 1.4.5
+ fs-constants: 1.0.0
+ inherits: 2.0.4
+ readable-stream: 3.6.2
+
+ teex@1.0.1:
+ dependencies:
+ streamx: 2.28.0
+ transitivePeerDependencies:
+ - bare-abort-controller
+ - react-native-b4a
+
+ text-decoder@1.2.7:
+ dependencies:
+ b4a: 1.9.0
+ transitivePeerDependencies:
+ - react-native-b4a
+
+ through@2.3.8: {}
+
+ throughput@1.0.2: {}
+
+ thunky@1.1.0: {}
+
+ timeout-refresh@1.0.3:
+ optional: true
+
tiny-invariant@1.3.3: {}
tinyglobby@0.2.16:
@@ -8647,6 +9852,22 @@ snapshots:
dependencies:
is-number: 7.0.0
+ torrent-discovery@11.0.21(supports-color@10.2.2):
+ dependencies:
+ bittorrent-dht: 11.0.12(supports-color@10.2.2)
+ bittorrent-lsd: 2.0.3(supports-color@10.2.2)
+ bittorrent-tracker: 11.2.3(supports-color@10.2.2)
+ debug: 4.4.3(supports-color@10.2.2)
+ run-parallel: 1.2.0
+ transitivePeerDependencies:
+ - bare-abort-controller
+ - react-native-b4a
+ - supports-color
+
+ torrent-piece@4.0.1:
+ dependencies:
+ uint8-util: 2.3.2
+
ts-api-utils@2.5.0(typescript@5.9.3):
dependencies:
typescript: 5.9.3
@@ -8667,6 +9888,10 @@ snapshots:
optionalDependencies:
fsevents: 2.3.3
+ tunnel-agent@0.6.0:
+ dependencies:
+ safe-buffer: 5.2.1
+
tw-animate-css@1.4.0: {}
type-check@0.4.0:
@@ -8721,6 +9946,8 @@ snapshots:
typescript@5.9.3: {}
+ uint8-util@2.3.2: {}
+
unbox-primitive@1.1.0:
dependencies:
call-bound: 1.0.4
@@ -8740,6 +9967,11 @@ snapshots:
universalify@2.0.1: {}
+ unordered-array-remove@1.0.2: {}
+
+ unordered-set@2.0.1:
+ optional: true
+
unrs-resolver@1.11.1:
dependencies:
napi-postinstall: 0.3.4
@@ -8794,8 +10026,37 @@ snapshots:
optionalDependencies:
'@types/react': 19.2.14
+ ut_metadata@5.0.1(supports-color@10.2.2):
+ dependencies:
+ bencode: 4.0.1
+ bitfield: 5.0.1
+ debug: 4.4.3(supports-color@10.2.2)
+ uint8-util: 2.3.2
+ transitivePeerDependencies:
+ - supports-color
+
+ ut_pex@5.0.2:
+ dependencies:
+ bencode: 4.0.1
+ compact2string: 1.4.1
+ string2compact: 2.0.2
+
+ utf-8-validate@6.0.6:
+ dependencies:
+ node-gyp-build: 4.8.4
+ optional: true
+
util-deprecate@1.0.2: {}
+ utp-native@2.5.3:
+ dependencies:
+ napi-macros: 2.2.2
+ node-gyp-build: 4.8.4
+ readable-stream: 3.6.2
+ timeout-refresh: 1.0.3
+ unordered-set: 2.0.1
+ optional: true
+
vary@1.1.2: {}
victory-vendor@36.9.2:
@@ -8817,6 +10078,62 @@ snapshots:
weak-lru-cache@1.2.2: {}
+ web-streams-polyfill@3.3.3: {}
+
+ webrtc-polyfill@1.2.2:
+ dependencies:
+ node-datachannel: 0.32.3
+
+ webtorrent@3.0.21(bare-url@2.5.4)(supports-color@10.2.2):
+ dependencies:
+ '@silentbot1/nat-api': 0.4.9(supports-color@10.2.2)
+ '@thaunknown/simple-peer': 10.1.2(supports-color@10.2.2)
+ '@webtorrent/http-node': 1.3.0
+ addr-to-ip-port: 2.0.0
+ bitfield: 5.0.1
+ bittorrent-dht: 11.0.12(supports-color@10.2.2)
+ bittorrent-protocol: 5.0.9(supports-color@10.2.2)
+ cache-chunk-store: 3.2.2
+ chunk-store-iterator: 1.0.4
+ cpus: 1.0.3
+ create-torrent: 6.1.3
+ cross-fetch-ponyfill: 1.0.3
+ debug: 4.4.3(supports-color@10.2.2)
+ escape-html: 1.0.3
+ fs-chunk-store: 5.0.1(bare-url@2.5.4)
+ fsa-chunk-store: 1.3.0
+ immediate-chunk-store: 2.2.0
+ join-async-iterator: 1.1.1
+ load-ip-set: 3.0.2
+ lt_donthave: 2.0.7(supports-color@10.2.2)
+ memory-chunk-store: 1.3.5
+ mime: 3.0.0
+ once: 1.4.0
+ parse-torrent: 11.0.24
+ pump: 3.0.4
+ queue-microtask: 1.2.3
+ random-iterate: 1.0.1
+ range-parser: 1.3.0
+ run-parallel: 1.2.0
+ run-parallel-limit: 1.1.0
+ speed-limiter: 1.0.2
+ streamx: 2.28.0
+ throughput: 1.0.2
+ torrent-discovery: 11.0.21(supports-color@10.2.2)
+ torrent-piece: 4.0.1
+ uint8-util: 2.3.2
+ unordered-array-remove: 1.0.2
+ ut_metadata: 5.0.1(supports-color@10.2.2)
+ ut_pex: 5.0.2
+ optionalDependencies:
+ utp-native: 2.5.3
+ transitivePeerDependencies:
+ - bare-abort-controller
+ - bare-buffer
+ - bare-url
+ - react-native-b4a
+ - supports-color
+
which-boxed-primitive@1.1.1:
dependencies:
is-bigint: 1.1.0
@@ -8848,6 +10165,9 @@ snapshots:
is-weakmap: 2.0.2
is-weakset: 2.0.4
+ which-runtime@1.4.0:
+ optional: true
+
which-typed-array@1.1.20:
dependencies:
available-typed-arrays: 1.0.7
@@ -8876,13 +10196,13 @@ snapshots:
'@cloudflare/workerd-linux-arm64': 1.20261001.1
'@cloudflare/workerd-windows-64': 1.20261001.1
- wrangler@4.147.0(@types/node@20.19.39):
+ wrangler@4.147.0(@types/node@20.19.39)(bufferutil@4.1.0)(utf-8-validate@6.0.6):
dependencies:
'@cloudflare/kv-asset-handler': 0.5.0
'@cloudflare/unenv-preset': 2.16.2(unenv@2.0.0-rc.24)(workerd@1.20261001.1)
blake3-wasm: 2.1.5
esbuild: 0.28.1
- miniflare: 5.20261001.0-alpha(@types/node@20.19.39)
+ miniflare: 5.20261001.0-alpha(@types/node@20.19.39)(bufferutil@4.1.0)(utf-8-validate@6.0.6)
path-to-regexp: 6.3.0
unenv: 2.0.0-rc.24
workerd: 1.20261001.1
@@ -8899,7 +10219,24 @@ snapshots:
string-width: 5.1.2
strip-ansi: 7.2.0
- ws@8.21.0: {}
+ wrappy@1.0.2: {}
+
+ ws@8.21.0(bufferutil@4.1.0)(utf-8-validate@6.0.6):
+ optionalDependencies:
+ bufferutil: 4.1.0
+ utf-8-validate: 6.0.6
+
+ ws@8.22.0(bufferutil@4.1.0)(utf-8-validate@6.0.6):
+ optionalDependencies:
+ bufferutil: 4.1.0
+ utf-8-validate: 6.0.6
+
+ xml2js@0.6.2:
+ dependencies:
+ sax: 1.6.1
+ xmlbuilder: 11.0.1
+
+ xmlbuilder@11.0.1: {}
yallist@3.1.1: {}
diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml
@@ -8,11 +8,21 @@ packages:
- umtool
allowBuilds:
+ # The seeder's WebTorrent (release 21 D4b): node-datachannel is WebRTC in
+ # node (a prebuilt binary, fetched by prebuild-install). The rest have pure-JS
+ # fallbacks (ws's bufferutil/utf-8-validate) or are optional (utp-native: uTP;
+ # without it the seeder speaks TCP and WebRTC), and ip-set's script is only a
+ # package-manager check.
+ bufferutil: false
esbuild: true
+ ip-set: false
lmdb: true
msgpackr-extract: true
+ node-datachannel: true
sharp: true
unrs-resolver: true
+ utf-8-validate: false
+ utp-native: false
workerd: false
ignoredBuiltDependencies:
diff --git a/scripts/archilyzer-ops.mjs b/scripts/archilyzer-ops.mjs
@@ -213,6 +213,13 @@ const ACTIONS = [
// Chosen media files of ONE archive.org item ({slug, item, files: [...] |
// match: "<regex>", dryRun?}): one job, one file at a time, paced.
"import-archive-org",
+ // A channel's held videos given their media from a LOCAL archive — a
+ // directory, a zip read in place, a 7z ({slug, source, items?, match?,
+ // createRecords?, replace?, dryRun?}): one job, nothing fetched.
+ "attach-media",
+ // A channel's saved containers remuxed losslessly into browser-playable
+ // copies, one torrent each ({slug, ids?, trackers?, root?, dryRun?}).
+ "prepare-playable",
// A podcast channel's records completed from its RSS feed ({slug, dryRun?}):
// one fetch of the feed, no media.
"feed-metadata",
@@ -765,6 +772,26 @@ export function usage() {
" queue; a low disk or a rate limit stops it, and running the same body",
" again resumes — saved videos are skipped.",
"",
+ 'attach-media copies each held video\'s file out of a LOCAL archive into the',
+ ' saved-video store, as its source container (nothing fetched): {"slug",',
+ ' "source"} — an absolute path to a directory, a .zip (read in place) or a',
+ ' .7z. The id is the folder\'s trailing "(<id>)", else the file\'s yt-dlp',
+ ' suffix; "items": [{"id", "path"}] names exact files (path inside the',
+ ' source), "match" narrows the folders by regex. "createRecords": true',
+ ' writes a record for a video the channel does not hold; "replace": true',
+ ' re-attaches over a saved container. "dryRun": true logs each folder\'s',
+ " class (attach, not-held, already-attached, lost, no-media, unmatched,",
+ " ambiguous) and the held videos with no media in it, and writes nothing.",
+ " The log ends with a summary: line; a re-run resumes.",
+ "",
+ 'prepare-playable remuxes each of a channel\'s saved containers, losslessly',
+ ' (-c copy), into a browser-playable mp4 (+faststart) or webm, and makes',
+ ' one single-file torrent per copy, under playable/ beside the saved-video',
+ ' store: {"slug"}. "ids" narrows it; "trackers": [...] is each .torrent\'s',
+ ' announce list (none by default; the infohash does not depend on it);',
+ ' "root" names another playable root. A video prepared from the same',
+ ' source (by sha256) is skipped; "dryRun": true logs each decision.',
+ "",
'fetch-windows fetches clip windows, one paced job per platform queue',
' (YouTube and Rumble side by side): {"siteId"} fetches every window the',
' site\'s published reports cite and the disk does not hold; {"items":',
diff --git a/settings.json.example b/settings.json.example
@@ -207,5 +207,14 @@
"previewBranch": "preview",
"hub": "off",
"homepage": "off"
+ },
+ "seeder": {
+ "sites": [],
+ "trackers": [],
+ "maxUploadKiBps": 0,
+ "maxConnections": 50,
+ "pollSeconds": 60,
+ "standbyAfterSeconds": 600,
+ "bindInterface": ""
}
}
