# Plan — a full sweep of a large Rumble channel must be paced, or it never completes

> **Superseded by `plans/release-5.md` slice R** (shipped on `one-core/r5-rumble`, 2026-09-24). The
> release plan corrects this one: four arg-builders plus the unargued probe, not one `configArgs`;
> impersonation for every Rumble spawn; a 429 mid-sweep is "incomplete" and falls back to the paged
> walk. Kept for its 2026-09-24 findings; do not implement from it.

**Found 2026-09-24** (operator: "job `01M3AVGZC5EZ9GCD04R9QW6NWX` keeps trying and failing a full
playlist fetch … no sync has happened in 44 days"). Facts verified on the live corpus and
`4130aca1`:

- `the-quartering-rumble` (`https://rumble.com/c/TheQuartering`, 7,866 listed / 7,871 archived)
  has `lastFullSweepAt: null`, so `isFullSweepDue` (`common/jobs/deepSync.ts:33-40`) is true on
  every sync and `sync()` (`common/ytdlp/runYtdlp.ts:1333-1336`) always takes `syncFullSweep`,
  never `syncPaged`.
- The sweep's one `--flat-playlist` spawn walks the channel's listing pages back to back; Rumble
  answers **HTTP 429 on page 155** (job log line 173). The patched extractor
  (`~/Projects/yt-dlp-patched`, the pipx editable install the editor runs) logs the error and
  finishes with 3,846 entries, but yt-dlp exits 1; `enumeratePlaylistUrls`
  (`runYtdlp.ts:352-361`) throws on any exit other than 0/101 and discards stdout. Nothing stamps
  `lastSyncedAt` (2026-08-11) or `lastFullSweepAt`; the next sync is a sweep again. Three
  attempts on 2026-09-24 (16:05, 20:20, 23:20 UTC), identical.
- **Tolerating exit 1 is the wrong fix.** The 3,846-entry listing is half the channel. The shrink
  guard (`common/controller/acceptListing.ts`, `fullSweepShrinkGuardPercent` 10 %, floor 25)
  would reject it once, and a second identical partial read — the same 429 at the same page —
  would *confirm* a mass deletion and truncate the stored `playlist`. The throw is what prevents
  that today.

**Step 1, applied by the operator 2026-09-24 (no code):** `fullSweepIntervalMinutes: 0` on the
channel, through `pnpm ops channel-config` (the `patchChannelConfig` writer), so every sync is
the paged walk (`--lazy-playlist -I start:end`, a few newest pages, stops at the first archived
hit) and new videos arrive on the next cadence. Cost until step 2: no maybe-missing detection for
this channel.

**Step 2, a small slice after one-core Phase 3 release 4** (it edits `runYtdlp.ts`, in slice W's
file set, so it lands after W merges):

1. **Pace the enumeration for Rumble.** Add `--sleep-requests <s>` to the sweep spawn when the
   channel's platform is Rumble (a per-platform table beside `downloadQueueKey`, not a per-channel
   knob; the value is an operator setting under `syncScheduler`, default ~1 s). Pacing applies to
   every page request, which is exactly where the 429 comes from.
2. **A sweep that ends on a 429 is "incomplete", not "failed" and not "a listing".** Detect the
   429 in stderr (or the non-zero exit with a partial stdout) and record it on the roster as an
   incomplete observation: it must not enter `acceptListing` as a reading (so two identical
   partial walks can never confirm a deletion), and it must not stamp `lastFullSweepAt`. Log one
   line naming the page it reached.
3. **Do not re-sweep on a cooldown.** After an incomplete sweep, the next syncs are paged walks
   until a per-platform cooldown (reuse the rate-limit cooldown the download runner already keeps
   for YouTube 429s, keyed `platform:rumble`) has elapsed — today a failed sweep is retried at
   every cadence, each one another 155-page walk that keeps the limit tripped.
4. **Tests:** unit for the incomplete verdict (partial + 429 → no `acceptListing` call, no
   stamp), the cooldown gate, and the pacing arg on a Rumble channel only; the `runYtdlp` e2e
   fixtures with a fake yt-dlp that exits 1 after N pages.
5. **Rollout:** after the slice is live, restore the channel's `fullSweepIntervalMinutes` (delete
   the override) and watch one paced sweep complete; the first complete listing after 44 days may
   legitimately shrink — the two-observation guard handles that as designed.

## Found next, 2026-09-24 19:36 — Rumble's embed endpoint answers 403 to every download

Step 1 worked: job `01M3AWDQ4RZD6CV93BJPREQVPX` ran the paged walk (`--lazy-playlist -I 1:50`),
listed the newest page and found new videos. The first download died on
`[RumbleEmbed] … Unable to download JSON metadata: HTTP Error 403: Forbidden`
(`https://rumble.com/embedJS/u3/`, `rumble.py:168`), the managed download aborted as "network",
and the sync failed on that one video. **It is not that video and not the patch:** the
`leaflit-rumble` and `rekietalaw-rumble` syncs at 16:05 UTC hit the same 403, and no retained
job log holds a successful Rumble archive line. Upstream: yt-dlp issue #17496 (opened 2026-08-20,
same error, same `2026.08.19`, labelled *impersonation* + *site-bug*, no fix), after #15089 /
#15148 / #15129 (Rumble behind Cloudflare, browser-only fingerprints); a June report says
`--impersonate chrome` worked for a while. The editor never passes `--impersonate`
(`grep -rn impersonate common editor` is empty) although the pipx venv has `curl_cffi` targets
(Chrome-133/136, Safari-18).

**Step 0 for the slice, ahead of pacing:** a per-platform yt-dlp args table (Rumble →
`--impersonate chrome`, and the pacing from step 2) applied in `configArgs`
(`common/ytdlp/runYtdlp.ts:221`) so every Rumble spawn — enumeration, metadata probe, download —
carries it; `downloadQueueKey` (`common/lib/queueKeys.ts:69`) already resolves the platform.
Verify first by hand, ONCE, metadata only (operator's rule: fetches go through Archilyzer;
this is the ask-first exception):
`yt-dlp --impersonate chrome --skip-download --print title https://rumble.com/v7fxzsc-lindsay-clancy-insane-twist-on-the-view-these-people-are-nuts.html`.
If that also 403s, Rumble is closed to this client until upstream moves and the slice waits;
the channels then need a "source unreachable" outcome rather than a failed sync per cadence.

Out of scope: tolerating exit 1 generally; changing the shrink guard; anything in the patched
extractor (the pagination fix is in and working; the 429 is Rumble's rate limit, not a bug).