Archilyzer · Source

archilyzer

Archilyzer
git clone https://archilyzer.pages.dev/source/archilyzer.git
Log | Files | Refs | README | LICENSE

commit ed6c3908ec1103042b304ada5eb5b2c5c2eb9d67
parent 41c2afc14d4b7433569113b05307fd524abf9710
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date:   Wed, 29 Jul 2026 21:01:33 -0400

Settle the pre-sweep decisions; abort the contention run on evidence

Three decisions had to be made before committing GPU-weeks, and two of
them stopped being judgement calls once the corpus was priced in chunks:

- Engine: qwen2.5:7b@8192 stays. ~25 days against gemma2's ~134 is a 5.4x
  cost decision, not taste — and gemma2 offloads 1,029 MB on this 8 GB
  card while qwen2.5 is fully resident, so part of its cost is the card.
  The ~6-8 h bake-off replicate matrix was descoped: the free census
  settled this without it.
- Tags: ON. 5-15% in the same pass against 44% as a later pass.
- digest-context-v1: STAYS v1, recorded next to the constant. The empty
  note already hashes stably, so Phase 1.5 notes invalidate only their own
  channels; only a change to the hashing scheme needs v2. Bumping it after
  the sweep would discard all 77k videos, so this needed deciding now.

THE CONTENTION MEASUREMENT WAS ATTEMPTED AND ABORTED, not skipped. An
unrelated containerised Playwright workload for another project was
running and respawning; over 90 s of sampling swap went 9.6 -> 18.4 GB of
19, MemAvailable fell to 1.7 GB, load hit 17, and the editor dev server
was killed mid-request. A run under that measures the swap. A number from
it would have been worse than no number — it would have gone into FACTS.md
and been planned against, which is the exact failure this session spent
its time correcting.

The attempt still confirmed, all timing-independent: `device` reaches the
yield through /api/workers (the 1b plumbing), the sweep resumes from
settings alone, the chunk-priced log is live ("8 chunks ... 9.14
chunks/audio-h"), tags generate through the real job path, and the
per-call load/prefill/decode breakdown is now IN the job log — the thing
that was previously unrecoverable.

Sampler and the aborted run's CSV are kept under plans/tools/ with the
void-conditions written down, so finishing this needs no rediscovery.

THE SWEEP IS NOT ARMED. Deliberately: the box is memory-starved and shared,
and arming starts GPU-weeks. Everything needed is in place and disarmed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Diffstat:
Mcommon/lib/digestContext-server.ts | 12++++++++++++
Mplans/STATE.md | 112++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---------
Aplans/tools/aborted-run-2026-07-30.csv | 45+++++++++++++++++++++++++++++++++++++++++++++
Aplans/tools/sample-contention.sh | 44++++++++++++++++++++++++++++++++++++++++++++
4 files changed, 201 insertions(+), 12 deletions(-)

diff --git a/common/lib/digestContext-server.ts b/common/lib/digestContext-server.ts @@ -41,6 +41,18 @@ export function digestContextPath(paths: Paths, channelSlug: string): string { // Hash the context inputs. Versioned by a literal prefix so the hashing scheme // itself can change later without colliding with old values. +// +// DECIDED 2026-07-29, BEFORE THE BACKFILL SWEEP: this stays at **v1**. +// +// The decision had to be made explicitly and in advance, because the prefix is +// folded into every section's `contextHash` and bumping it after the sweep would +// invalidate all ~77k videos — GPU-weeks thrown away by a one-character edit. +// +// v1 is correct because the EMPTY note already hashes stably. Adding a per-channel +// context note later (Phase 1.5) changes `note`, not the scheme, so it invalidates +// only the channels that get a note — which is the desired behaviour, not a +// problem to be versioned around. **Only a change to the hashing SCHEME itself +// needs v2** (a different algorithm, or folding in a second input). export function hashDigestContext(note: string): string { return createHash("sha1") .update(`digest-context-v1\n${note}`) diff --git a/plans/STATE.md b/plans/STATE.md @@ -3,7 +3,18 @@ The working memory for the local-AI derived-corpus work. Rewritten at the end of every session, before context is cleared. See [`README.md`](README.md) for the protocol. -**Last updated:** 2026-07-29 — **the backfill is at the starting line.** A corpus-wide sweep +**Last updated:** 2026-07-30 — **the cost model was wrong in its UNITS, and fixing +that dissolved the conflict between three measurements.** A chunk (one model call) +is the unit of work; chunk density varies 4× across the corpus, so +seconds-per-audio-hour is not a unit. Re-priced, 27 / 90 / 151 s/audio-hour all +agree to within ~2% and the corpus is **191,116 chunks → ~25 days idle to ~55 +contended, not 81**. The ~6–8 h bake-off replicate matrix was descoped as +unnecessary: the census cost zero GPU time and settled the model choice too. Tags +were generated for the first time ever (they had NEVER run) and cost 5–15%, not +100%. Two shipped bugs fixed. The contention measurement was attempted and +**aborted on evidence** — see below; it is the one thing still open. + +**Previously:** 2026-07-29 — **the backfill is at the starting line.** A corpus-wide sweep can now be started from one control, survives a server restart, can be paused without being lost, yields the GPU to transcription, and reports coverage and an ETA in the unit the work is actually priced in. Stages A–D of the "cheapest-first" plan are all in; see the decisions @@ -68,6 +79,53 @@ made configurable (`digest.yieldToCpuWorkers`, default off). That fix is plausib a bigger throughput lever than the model choice, and it is a logic error rather than physics. +**The contention measurement was ATTEMPTED AND ABORTED, and it is still open.** +Not descoped — attempted, aborted on evidence, and left with its tooling in place. +The box could not support it: an unrelated containerised Playwright workload for +another project was running (and respawning), and over the 90 s of sampling +`plans/tools/aborted-run-2026-07-30.csv` records **swap going 9.6 → 18.4 GB of +19 GB**, `MemAvailable` down to 1.7 GB, load average 3.4 → 17.0, and finally the +editor dev server dying mid-request with 28 s response times. A run under those +conditions measures the swap, which this document already warns about. **A number +from it would have been worse than no number**, since it would have gone into +FACTS.md and been planned against — the exact failure this session spent its time +correcting. + +What the attempt DID confirm in production, none of which is timing-dependent: + +- **`device` reaches the yield.** `GET /api/workers` reports `device: "cpu"` for + both CPU workers and no `device` for the GPU one, from the same `summary()` call + `transcriptionActivity()` reads. That was the missing plumbing in the 1b fix. +- **The sweep resumes from settings alone**, again: a restart re-launched + `digest-sweep` + `digest-channel-local` on the persisted scope with no cursor. +- **The chunk-priced log is live**: `Pass 1: 1 channel(s), 8 chunks / 1 + audio-hours to generate (~0.0 days at the measured rate, 9.14 chunks/audio-h)`. +- **Tags generate through the real job path** with 0 warnings. +- **The per-call breakdown is now IN the job log** — `in 5s wall (load 0.4s, + prefill 0.1s, decode 3s, engine 3.5s)`. This is the thing that was previously + unrecoverable: job logs recorded per-call seconds with no way to separate engine + from wall, and the last run's logs had already rotated away. + +**What remains unmeasured: the per-chunk wall/engine multiplier under real GPU +contention, and the live A/B of `yieldToCpuWorkers`.** The A/B is unit-tested in +both directions against this box's exact worker set +(`controller/digestYield.test.ts`), so what is missing is specifically a +production run with genuinely busy CPU workers — which needs eligible +transcription work, and the only work available is 2.3–4.6 GB multi-hour VODs that +would take hours on CPU-only workers. + +To finish it, on a box with nothing else running: + +1. `plans/tools/sample-contention.sh out.csv` — start the sampler first. +2. Enable only the two `device: "cpu"` workers (note: `transcripts/.workers/defaults.json` + OVERRIDES `settings.json`'s per-worker `enabled` at boot — an hour lost to that). +3. Queue transcription for a channel with media on disk (`cornbreadman` has 7). +4. Arm a scoped sweep, `sections: ["chapters","tags"]`, and read + `.jobs/<id>.log` for the yield line and the per-call breakdown. +5. Repeat with `digest.yieldToCpuWorkers: true`, changing NOTHING else. +6. Void any cell whose CSV shows `gpu_worker_busy` 0 throughout, swap growing, or + thermal throttling. + **The sweep was run for real, not just built.** Scoped to `teamrcn` (7 videos, 0.27 audio-hours) against ollama and the real corpus: it armed with its scope persisted, launched the per-channel job, generated 7 digests with correct @@ -112,7 +170,7 @@ borrowed digest is labelled as borrowed. | 4 · Search indexing | not started | | | 5 · Auto-queue | not started | | | 6 · Ollama `/ask` provider | not started | **No dependencies.** | -| 7 · Tags + chat highlights | not started | Tag generation works; nothing consumes it. | +| 7 · Tags + chat highlights | not started | Tag generation is now VERIFIED WORKING (it had never run once — 0 of 109 sidecars). 11 videos, 0 warnings, metrics in `digest-validate.ts`, e2e in `digest.spec.ts`. Nothing consumes it yet. Known gap for this phase: only 2.4% cross-video tag reuse, and `antisemitism`/`anti-semitism` are two tags — the vocabulary does not converge, so a consumer needs normalization. | | 8 · Visibility policy | not started | | | 9 · Attribution + quote filtering | not started | | | 10 · Lead with the derived corpus | not started | | @@ -135,22 +193,52 @@ Exploration for Phase 2 found the planning docs describing more unbuilt work tha line and every cross-channel surface gets a counter it is already paying for. Left unchanged here deliberately: it is Phase 2.5's to take, not a digest-corpus side effect. +### Pre-sweep decisions — settled 2026-07-29 + +All three of the questions that had to be answered *before* committing GPU-weeks +are now answered, and two of them stopped being judgement calls. + +| Decision | Settled | Why | +| --- | --- | --- | +| **Engine** | `qwen2.5:7b` @ 8192, chunk-local — already the shipped default | The census makes it arithmetic, not taste: **~25 days against gemma2's ~134, a 5.4× cost decision.** gemma2 also **offloads 1,029 MB** at 8192 ctx on this 8 GB card while qwen2.5 (5.12 GB) is fully resident, so part of its cost is the card. gemma2 becomes a targeted per-channel re-run later; regeneration is supported and per-channel. **No new bake-off matrix was needed** — the free census settled it. | +| **Tags** | **ON** for the first pass | Measured surcharge in the same pass is **5–15%**; deferred to a separate pass it is **44%** of a whole chapters pass. Including them now is 3–9× cheaper than adding them later, and avoids a second corpus-wide pass entirely. | +| **`digest-context-v1`** | **Stays v1** | The empty note already hashes stably, so Phase 1.5 can add per-channel notes later and invalidate only those channels. Only a change to the hashing SCHEME needs v2. Recorded in `digestContext-server.ts` next to the constant, because bumping it after the sweep would discard all 77k videos. | + +**The sweep was NOT armed, deliberately.** Two reasons, and the first is the +blocking one: the box is currently running an unrelated containerised workload for +another project and sat at **18.4 GB of 19 GB swap** with 1.7 GB RAM available +during the attempted measurement — the editor dev server itself was killed. Arming +a corpus-wide sweep on that box would be GPU-weeks of work started into memory +starvation. Second, the plan's own gate for arming was the contention measurement, +and that cell is void. Everything needed to arm is in place and disarmed: +`sweepEnabled: false`, `sections: ["chapters"]`, transcriptions paused, worker +defaults restored. + +Still unverified as a result: **kill-and-restart mid-sweep on a MULTI-CHANNEL +run.** Single-channel resume-with-zero-rework is verified twice over; the +multi-channel case is the one the plan wanted and it needs an armed sweep. + **Recommended next**, in the order the measurements argue for: -1. **MEASURE THE GPU YIELD IN PRODUCTION** — still the most valuable single action, but - NOT "second by a factor of fifty": that framing came from the retracted 2.8x and is - withdrawn. The re-priced spread is ~25 days idle to ~55 contended, so contention is - worth up to ~30 days against sharing's ~2 — large, and no longer fabricated. - Measure **seconds per CHUNK**, not per audio-hour, with a scoped sweep run twice - differing only in `digest.yieldToCpuWorkers`, and compute wall/engine from - `.jobs/<id>.meta.json` against the summed per-call lines in `.jobs/<id>.log` - (`digestApps.ts` now records load/prefill/decode per call, which is what makes the - result interpretable rather than just a ratio). +1. **MEASURE THE GPU YIELD IN PRODUCTION, ON A BOX WITH NOTHING ELSE ON IT.** Still the + most valuable single action, but NOT "second by a factor of fifty": that framing came + from the retracted 2.8x and is withdrawn. The re-priced spread is ~25 days idle to ~55 + contended, so contention is worth up to ~30 days against sharing's ~2 — large, and no + longer fabricated. **Attempted 2026-07-30 and aborted on evidence** (swap 9.6 → 18.4 GB + of 19; the editor was killed). Full procedure, tooling and void-conditions are in the + contention section above; measure **seconds per CHUNK**, never per audio-hour. + `digestApps.ts` now logs load/prefill/decode per call, which is what makes the result + interpretable rather than just a ratio. **The box being quiet is a precondition, not a + nicety** — check `free`, `/proc/pressure/memory` and `docker ps` first. 2. **Run the sweep for a day and watch it.** The launcher, resume, pause and coverage readouts are all built and unit-tested but have not driven a real multi-channel run. Kill the editor mid-run and confirm it resumes with zero rework (eligibility is re-derived from disk, so the correct result is zero). -3. **Decide the `digest-context-v1` hash-prefix question before sweeping.** This is the real +3. ~~**Decide the `digest-context-v1` hash-prefix question before sweeping.**~~ + **DONE — stays v1**, recorded next to the constant in `digestContext-server.ts` and in + the pre-sweep decisions table above. Original reasoning retained below. + + This is the real 1.5 gate, and it is a decision, not work: bumping the prefix invalidates the entire corpus, so it must happen before 80 GPU-days go in, or not at all. 4. **Then Phase 4 search indexing / Phase 7 tags,** which are starved until coverage exists. diff --git a/plans/tools/aborted-run-2026-07-30.csv b/plans/tools/aborted-run-2026-07-30.csv @@ -0,0 +1,45 @@ +ts,gpu_busy_pct,vram_used,temp_c,power_w,sclk_hz,mem_avail_kb,swap_used_kb,psi_mem_full_avg10,loadavg1,ollama_models,gpu_worker_busy,cpu_workers_busy,active_jobs,digest_jobs +2026-07-30T00:50:48Z,3,759037952,41,10,28000000,2325540,10028300,1.69,3.38,,,,, +2026-07-30T00:50:51Z,6,759037952,41,10,28000000,8907524,8763332,3.38,4.23,,,,, +2026-07-30T00:50:53Z,12,759037952,41,10,12000000,8591352,8796156,4.03,4.23,,,,, +2026-07-30T00:50:55Z,1,759037952,41,10,10000000,8334660,8871292,4.75,4.23,,,,3,1 +2026-07-30T00:51:09Z,1,759037952,40,10,34000000,5615944,9493212,5.99,7.18,,0,0,4,2 +2026-07-30T00:51:14Z,0,759037952,41,9,128000000,5340952,9567000,4.94,8.45,,0,0,4,2 +2026-07-30T00:51:18Z,1,759037952,40,9,8000000,5220568,9545548,3.49,8.09,,0,0,4,2 +2026-07-30T00:51:27Z,11,759058432,41,8,43000000,4656868,9628916,2.05,9.68,,,,, +2026-07-30T00:51:45Z,99,5771677696,50,135,2324000000,3570452,10519580,6.01,9.56,qwen2.5:7b,0,0,4,2 +2026-07-30T00:51:52Z,99,5771681792,54,135,2381000000,3111368,10491620,4.12,10.04,qwen2.5:7b,0,0,4,2 +2026-07-30T00:52:00Z,99,5773705216,49,72,2204000000,2872796,10807284,4.90,9.71,qwen2.5:7b,0,0,4,2 +2026-07-30T00:52:07Z,23,5771706368,54,103,917000000,2424508,11061636,5.74,10.27,qwen2.5:7b,0,0,4,2 +2026-07-30T00:52:11Z,4,5771706368,46,10,39000000,6889064,10461404,5.02,10.09,qwen2.5:7b,,,, +2026-07-30T00:52:13Z,3,5771706368,45,9,28000000,6888644,10539080,4.83,10.09,qwen2.5:7b,,,, +2026-07-30T00:52:15Z,5,5771706368,44,10,73000000,6620832,10543972,4.32,9.68,qwen2.5:7b,0,0,3,1 +2026-07-30T00:52:24Z,14,5771698176,43,9,39000000,4738716,10628680,2.25,10.51,qwen2.5:7b,0,0,3,1 +2026-07-30T00:52:28Z,18,5771698176,44,9,47000000,4203800,10687472,2.43,11.91,qwen2.5:7b,0,0,4,2 +2026-07-30T00:52:34Z,14,5771698176,43,9,53000000,4102300,10640092,2.99,11.35,qwen2.5:7b,0,0,, +2026-07-30T00:52:44Z,11,5771698176,43,10,34000000,3063692,10155400,1.59,10.54,qwen2.5:7b,0,0,, +2026-07-30T00:52:56Z,1,5771702272,43,9,18000000,2842816,10257860,1.86,12.41,qwen2.5:7b,0,0,4,2 +2026-07-30T00:53:04Z,91,5771685888,49,79,2494000000,2732520,10472036,2.33,12.14,qwen2.5:7b,0,0,4,2 +2026-07-30T00:53:12Z,99,5771689984,54,118,2381000000,2620844,11194664,7.16,13.59,qwen2.5:7b,0,0,4,2 +2026-07-30T00:53:17Z,99,5771689984,55,134,2316000000,4812096,10286832,4.98,13.71,qwen2.5:7b,0,0,4,2 +2026-07-30T00:53:21Z,99,5771702272,51,78,2336000000,4533392,10299024,3.48,12.85,qwen2.5:7b,0,0,4,2 +2026-07-30T00:53:30Z,87,5771698176,52,78,2420000000,4314652,10241652,1.43,11.34,qwen2.5:7b,0,0,4,2 +2026-07-30T00:53:34Z,99,5771702272,56,135,2317000000,4232296,10283220,1.32,11.34,qwen2.5:7b,0,1,4,2 +2026-07-30T00:53:38Z,97,5771694080,52,82,2644000000,4124996,10147408,1.06,10.67,qwen2.5:7b,0,0,5,2 +2026-07-30T00:53:47Z,99,5771694080,56,134,2343000000,3517920,9606984,5.68,11.29,qwen2.5:7b,0,0,, +2026-07-30T00:54:01Z,1,5771694080,45,10,39000000,2315108,10728740,10.04,11.15,qwen2.5:7b,0,0,5,2 +2026-07-30T00:54:14Z,99,5771694080,57,135,2303000000,2080628,12997824,18.98,13.03,qwen2.5:7b,0,0,, +2026-07-30T00:54:27Z,8,5771694080,46,10,18000000,2319340,15459608,21.71,13.80,qwen2.5:7b,0,0,, +2026-07-30T00:54:39Z,5,5771698176,47,11,26000000,2177652,16945948,15.95,15.17,qwen2.5:7b,0,0,, +2026-07-30T00:54:50Z,7,5771698176,46,10,40000000,2644936,18039460,7.67,16.39,qwen2.5:7b,,,, +2026-07-30T00:55:09Z,0,5771698176,45,10,60000000,1473204,18324000,3.83,16.21,qwen2.5:7b,,,, +2026-07-30T00:55:27Z,0,5771698176,45,10,142000000,1170180,17947780,10.95,17.59,qwen2.5:7b,,,, +2026-07-30T00:55:45Z,13,5771698176,44,9,48000000,1654800,19626676,9.98,17.86,qwen2.5:7b,,,, +2026-07-30T00:56:03Z,0,5771698176,45,10,707000000,1136736,19056176,8.31,16.56,qwen2.5:7b,,,, +2026-07-30T00:56:21Z,6,5771702272,44,9,37000000,1828300,19756104,6.52,16.67,qwen2.5:7b,,,, +2026-07-30T00:56:39Z,15,5771702272,44,11,24000000,2031176,20199132,4.86,19.10,qwen2.5:7b,,,, +2026-07-30T00:56:57Z,4,5771702272,43,10,23000000,2431360,20311724,4.09,18.45,qwen2.5:7b,,,, +2026-07-30T00:57:16Z,99,5771702272,56,135,2345000000,1860408,19254700,3.44,16.95,qwen2.5:7b,,,, +2026-07-30T00:57:34Z,2,5771702272,46,9,83000000,1761284,19986096,3.73,19.24,qwen2.5:7b,,,, +2026-07-30T00:57:52Z,6,5771702272,45,11,20000000,1814964,20190332,4.80,21.56,qwen2.5:7b,0,0,, +2026-07-30T00:58:06Z,99,5771702272,56,135,2309000000,1783404,19585500,6.58,18.78,qwen2.5:7b,,,, diff --git a/plans/tools/sample-contention.sh b/plans/tools/sample-contention.sh @@ -0,0 +1,44 @@ +#!/bin/bash +# Sampler for the digest-vs-transcription contention measurement (the "named +# test" in STATE.md). Writes one CSV row every 2 s. +# +# It samples the SAME TWO SIGNALS `controller/digestYield.ts` reads, through the +# endpoints that expose them — `GET /api/workers` (built from the same +# `summary()` call, so `busy` and `device` are exactly what the yield sees) and +# `GET /api/jobs/active` — beside the card's own view. That is what lets a run be +# declared VOID rather than quietly believed: if `gpu_worker_busy` was 0 +# throughout, the cell measured nothing. +# +# It also samples the two things that make such a run LIE: +# - swap_used_kb / mem_avail_kb / psi_mem_full_avg10 — a run that swaps +# measures the swap. This is not hypothetical; see +# aborted-run-2026-07-30.csv, where swap went 9.6 -> 18.4 GB of 19 and the +# editor died mid-request. +# - temp_c / power_w / sclk_hz — an RX 6600 throttles over 2-3 GPU-hours, and +# a throttled second cell would look like a regression in the first. +# +# Usage: ./sample-contention.sh out.csv (Ctrl-C to stop; assumes card1) +OUT="$1"; CARD=/sys/class/drm/card1/device; HW=$CARD/hwmon/hwmon2 +echo "ts,gpu_busy_pct,vram_used,temp_c,power_w,sclk_hz,mem_avail_kb,swap_used_kb,psi_mem_full_avg10,loadavg1,ollama_models,gpu_worker_busy,cpu_workers_busy,active_jobs,digest_jobs" > "$OUT" +while true; do + ts=$(date -u +%Y-%m-%dT%H:%M:%SZ) + gpu=$(cat $CARD/gpu_busy_percent 2>/dev/null||echo -1) + vram=$(cat $CARD/mem_info_vram_used 2>/dev/null||echo -1) + temp=$(( $(cat $HW/temp1_input 2>/dev/null||echo 0) / 1000 )) + pw=$(( $(cat $HW/power1_average 2>/dev/null||echo 0) / 1000000 )) + sclk=$(cat $HW/freq1_input 2>/dev/null||echo 0) + memav=$(awk '/MemAvailable/{print $2}' /proc/meminfo) + swt=$(awk '/SwapTotal/{print $2}' /proc/meminfo); swf=$(awk '/SwapFree/{print $2}' /proc/meminfo) + swu=$((swt-swf)) + psi=$(awk -F'[= ]' '/^full/{print $3}' /proc/pressure/memory) + la=$(awk '{print $1}' /proc/loadavg) + om=$(curl -s --max-time 2 localhost:11434/api/ps | jq -r '[.models[]?.name]|join(";")' 2>/dev/null) + w=$(curl -s --max-time 8 localhost:3001/api/workers) + gb=$(echo "$w" | jq -r '[.workers[]?|select(.kind=="local" and ((.device // "")|ascii_downcase)!="cpu")|select(.busy)]|length' 2>/dev/null||echo -1) + cb=$(echo "$w" | jq -r '[.workers[]?|select(.kind=="local" and ((.device // "")|ascii_downcase)=="cpu")|select(.busy)]|length' 2>/dev/null||echo -1) + aj=$(curl -s --max-time 8 localhost:3001/api/jobs/active) + n=$(echo "$aj" | jq -r '[.jobs[]?|select(.status=="running")]|length' 2>/dev/null||echo -1) + dj=$(echo "$aj" | jq -r '[.jobs[]?|select(.status=="running" and (.kind|test("digest")))]|length' 2>/dev/null||echo -1) + echo "$ts,$gpu,$vram,$temp,$pw,$sclk,$memav,$swu,$psi,$la,$om,$gb,$cb,$n,$dj" >> "$OUT" + sleep 2 +done