commit f985d12aa555e32054e059d01cce64a971c070df
parent 1151b0317b7448e064b9639b44af5d137b473329
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date: Wed, 30 Sep 2026 09:20:05 -0400
docs: PUBLISH.md's source-mirror section gains the history pages — the table row, the install line for stagit, what the step does with it (the allowlist, the post-pass, the gate, the manifest block, the render cache), its size at 1,872 commits and the file limit ahead; the withdrawal and the deploy check name the history; the homepage's Install and FAQ mention it; the editor and homepage [Unreleased] bullets
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Diffstat:
5 files changed, 86 insertions(+), 12 deletions(-)
diff --git a/PUBLISH.md b/PUBLISH.md
@@ -26,7 +26,7 @@ server-side code, servable by anything:
|---|---|---|---|
| A **site** | The export app over the channels one site selects: search, transcripts, charts, downloads. One corpus can publish several. | `export/out` (docker mode: `export/.export-builds/<siteId>/out`) | `transcripts/sites/<id>/site.json` ([SITE.md](SITE.md)) |
| The **hub** | The export app in hub mode: federated search across the family of sites. | `export/out` | `transcripts/sites/_homepage/homepage.json` |
-| The **homepage** | The project's own site (`homepage/`), with the source mirror, its raw tree and the source tarball. | `homepage/out` | `~/.config/archilyzer/` (the source mirror's two operator files) |
+| The **homepage** | The project's own site (`homepage/`), with the source mirror, its raw tree, its history pages and the source tarball. | `homepage/out` | `~/.config/archilyzer/` (the source mirror's two operator files) |
The hub and the homepage are two different Pages projects: the hub deploys to the
project `homepage.json` names (e.g. `archilyzer-hub`) and is refused `archilyzer`,
@@ -104,6 +104,7 @@ The homepage carries the project's own source, read-only, as static files:
| `/source/` | The page: the clone command, the mirror head, the private `main` it reflects |
| `/source/archilyzer.git/` | A git repository for git's **dumb HTTP** protocol: `git clone https://archilyzer.pages.dev/source/archilyzer.git` |
| `/source/tree/` | Every tracked file of `main`, raw, with a generated `index.html` per directory |
+| `/source/git/` | The history: `log.html`, a page per commit with its diff (`commit/<sha>.html`), `refs.html`, `files.html` (an index into the raw tree), `atom.xml` and `tags.xml` — rendered by **stagit** when it is installed (below) |
| `/downloads/archilyzer-source.tar.gz` + `snapshot.json` | The same tree without history |
| `/source/manifest.json` | What was published, from which private commit, audited how |
@@ -117,7 +118,8 @@ step falls back to `pipx run`, which needs the network).
before `next build`, and:
- the step removes the LAST publish from `homepage/public` (manifest first, then the
- mirror, the tree, the tarball, `snapshot.json` and the skip key) — it was audited
+ mirror, the tree, the history pages, the tarball, `snapshot.json` and the skip key,
+ and the history's render cache from the scratch root) — it was audited
under rules that may not be today's, and the refusal is the best evidence that they
are not. Any outcome but success does this once the rules are loaded (an audit hit,
a limit, a missing tool, a cancel, a crash); `--check` writes nothing, this included;
@@ -127,9 +129,12 @@ before `next build`, and:
source unless the skip key says that publish was made under today's rules and step
version, of today's `main`, by today's gitleaks, and is exactly the one in `out/`:
its mirror head, and a digest over every published file (the mirror, the tree, the
- manifest, the tarball, `snapshot.json` — sorted path, size and sha256, recomputed
- over `out/`), so a mixed or edited `out/` refuses too: "run `archilyzer build
- homepage` (it re-audits), then deploy". An `out/` whose `/source` page shows the empty
+ history pages, the manifest, the tarball, `snapshot.json` — sorted path, size and
+ sha256, recomputed over `out/`), so a mixed or edited `out/` refuses too: "run
+ `archilyzer build homepage` (it re-audits), then deploy". History pages that are
+ not the audited ones (edited, missing, or present when none were published) are
+ named on their own: "homepage/out's history pages (/source/git/) are not the ones
+ that were audited". An `out/` whose `/source` page shows the empty
state (`--no-source`) deploys as before; one with no `/source` page (a refused build)
does not.
@@ -175,7 +180,9 @@ What one publish does:
then re-run.`
5. The tree (`git archive` → `tar -x`, a page per directory; a tracked `index.html`,
a tracked `404.html` — Pages would serve it as HTML for every missing path below it —
- or a symlink is a refusal) and the tarball (`git archive --format=tar.gz -9`).
+ or a symlink is a refusal) and the tarball (`git archive --format=tar.gz -9`). Then
+ the history pages (stagit, below), into the same stage — so the file sweep of step 4
+ reads every one of them too.
6. Limits, then the install: link-safe (`common/bin/_publicFile.ts`), the manifest
removed first and written **last**, so a crash leaves the page's empty state,
never a manifest over a half-copied tree.
@@ -226,13 +233,77 @@ must be removed from history by hand.
**Tools.** `git filter-repo` (`pipx install git-filter-repo`; without it the step runs
`pipx run --spec git-filter-repo==2.47.0`, which needs the network on first use, and
without pipx it refuses with the install line). gitleaks is optional: without it the
-secret scan is skipped with a WARNING and the literal audit still runs.
-`archilyzer doctor` has a "source publish" block: which filter-repo would run,
-gitleaks, the two files (rule counts and modes, never contents) and the last publish.
+secret scan is skipped with a WARNING and the literal audit still runs. stagit is
+optional too (next section). `archilyzer doctor` has a "source publish" block: which
+filter-repo would run, stagit (its path, or not found), gitleaks, the two files (rule
+counts and modes, never contents) and the last publish.
The mirror's ids are deterministic for a given filter-repo version; an upgrade that
changes its rewriting changes every id (readers re-clone). The manifest records both
tools.
+**The history pages (`/source/git/`) are stagit's.** [stagit](https://codemadness.org/stagit.html)
+(C over libgit2) renders the scrubbed mirror as static pages: the log, a page per commit
+with its diffstat and diff, the refs, and two Atom feeds. It is an operator-installed
+tool like git-filter-repo — never vendored, never committed. Install it once (libgit2
+and its headers are the one dependency):
+
+```sh
+git clone git://git.codemadness.org/stagit && make -C stagit && cp stagit/stagit ~/.local/bin/
+```
+
+The step finds it as `STAGIT_BIN`, else `stagit` on `PATH`, else `~/.local/bin/stagit`
+(the editor's process needs `~/.local/bin` on its `PATH` for a pipx `git filter-repo`;
+for stagit the fallback covers it). **Without it the source is published without the
+history**: one line says so and how to install it, the manifest has no `history`
+block, and `/source/` shows no History links. A render that fails, or history pages
+that would break the host's limits, are the same, with a WARNING — never a failed
+build, and the next build tries again (a stagit present and no history never skips).
+What the step does with it:
+
+- It runs after the mirror is built and its objects audited: `stagit -c <cache> -u
+ https://archilyzer.pages.dev/source/git/ <the scrubbed bare clone>`. The clone is
+ named `archilyzer.git` (stagit names the repository after its directory); its
+ `description` is set to "Archilyzer" and its `url` to the clone URL, for stagit's
+ header. Neither file is published.
+- **Only an allowlist is published:** `log.html`, `files.html`, `refs.html`,
+ `atom.xml`, `tags.xml`, `commit/<sha>.html`, and a `style.css` the step writes from
+ `common/styles/tokens.css` (the homepage's two grounds, light and dark; diff
+ insertions in `--success`, deletions in `--destructive`). **stagit's per-file
+ pages (`file/…`) are not**: browsing is the raw tree, so every link into `file/` —
+ the Files index, the README and LICENSE links in the header, a diff's file names —
+ is rewritten to `../tree/<path>`.
+- Every `.html` page (never the feeds) gets exactly two additions: the homepage's
+ pre-paint theme script (the one `ThemeScript` emits, `lib/themeConfig.ts`), so a
+ page opens on the visitor's stored base or the homepage's dark default — without
+ JavaScript, `prefers-color-scheme` decides — and one line at the top, "Archilyzer ·
+ Source", linking to `/source/`. stagit's logo and favicon point at the site's
+ `/icons/icon-32.png`. Nothing else is changed; the pages are rewritten byte for byte.
+- **The gate reads every page**: they are in the stage the file sweep reads, so a
+ denied literal in one refuses the publish, named by its path (`file
+ source/git/commit/<sha>.html (contents, byte N)`) and never by its bytes. The pages
+ are rendered from objects the object sweep already read, so a hit there means
+ something stagit or the step added.
+- The manifest's `history` block has the log's href, the commit count, the head, the
+ file count and bytes, a sha256 over the pages, and stagit's identity (the sha256 of
+ its binary; it has no version flag). The skip key has stagit's identity and the
+ pages' digest.
+- **The render cache.** stagit's `-c` keeps a commit page once rendered, so a publish
+ renders only the new commits: 2 s with nothing new against 8 s for every page, at
+ 1,834 commits. The cache (the cache file, stagit's output, a key) is
+ `archilyzer-source-history/` in the scratch root (`ARCHILYZER_SOURCE_SCRATCH`, the OS
+ temp dir by default — on a tmpfs `/tmp` that is about 140 MB of memory).
+ It is used only under the same rules, step, filter-repo, stagit and header text, when
+ the commit it names is an ancestor of today's head, and after a run that finished;
+ one publish holds it at a time. `--force` renders every page again, `--check` never
+ touches it, and a refusal removes it.
+- **Its size, and the limit ahead.** At 1,872 commits (2026-09-30): 1,878 files,
+ 141.3 MB, the largest page 5.0 MB (stagit prints "Diff is too large, output
+ suppressed" past 1,000 files or 100,000 lines in one commit); the whole publish is
+ 4,396 files, and `homepage/out` 4,576. One file per commit counts against the step's
+ 15,000 (of Pages' 20,000): the history is dropped with a WARNING, never the publish,
+ when it would cross it. At the pace of September 2026 (about 100 commits a day) that
+ is some three months away.
+
**Cloudflare Pages, and the traps it sets.**
- **Never a path segment named `.git`.** wrangler's upload ignore list drops `**/.git`
diff --git a/editor/CHANGELOG.md b/editor/CHANGELOG.md
@@ -15,6 +15,7 @@
- **The hub URL hints say what the setting does now.** Settings' **Family hub URL** and a site's **Hub URL** no longer promise a Hub link in the header (it was removed): the value is published as `hubUrl` in each site's `/site.json` and `/corpus.json`, so the hub can tell its member sites. `SETTINGS.md` and `SITE.md` say the same.
- **A site can be left off the homepage and the hub.** A site's settings have a new checkbox, **List on the Archilyzer homepage and hub**, on by default (`listed` in `site.json`; only `false` is written). Turned off, the site still builds and deploys at its own URL as before, but the homepage has no card, chart series, `/stats` entry or recent item for it; the hub does not list it as a member, search it, or name it in its `corpus.json` and `llms.txt`; no other site's footer links it; and `channel-sites.json` and the homepage's `stats/` leave it out. A channel only unlisted sites carry is in none of the published totals, the homepage's headline numbers included; a channel a listed site also carries is counted under the listed site. The editor's own pages still show every site. It takes effect at the next homepage, hub and site builds.
- **The sidebar's site picker shows your site from the first paint.** It used to show "All sites" on every page and then jump to the site you had picked, and Dashboard and Channels came up in your site only after a `?site=` had been added to the address. The picked site is now kept in a cookie that the editor reads before it draws a page, so the picker, Dashboard and Channels open in it at once, and the address is left alone. A link that carries `?site=<id>` still opens that page in that site, without changing the one you picked; picking a site on such a page drops the `?site=` from the address. On a site's own pages (Charts, Publish, …) the picker still follows the page, and opening one still makes that site the picked one. The first time you open the editor after updating, a site picked before is moved into the cookie; the picker may show "All sites" for a moment that once. A site picked in one tab reaches the editor's other open tabs without a reload. **New channel** starts with the picked site ticked under its sites (or the site of a `?site=` link), including when it is opened from the editor's own links. Each editor keeps its own pick, as before, when several run on one machine on different ports.
+- **Building the homepage publishes the source's history too: every commit of main with its diff, at `/source/git/`, when stagit is installed.** `archilyzer build homepage`, the `/sites` Homepage jobs and `pnpm ops build-homepage` render the scrubbed mirror with stagit into a log, a page per commit, the refs and two Atom feeds; each page carries the homepage's grounds and one line back to `/source/`, its Files page links into the raw tree, and every page goes through the same gate as the mirror (a denied literal in one refuses the publish). stagit is installed once, outside the repository: `git clone git://git.codemadness.org/stagit && make -C stagit && cp stagit/stagit ~/.local/bin/` (or point `STAGIT_BIN` at it). Without it the build goes on without the history and says so in one line; a render that fails, or pages that would pass the host's limits, are a warning, never a failed build. `archilyzer doctor` shows where stagit is, beside git-filter-repo. The first build after this change publishes the source again (the step's version is 4). A render cache of about 140 MB is kept in the scratch root (`ARCHILYZER_SOURCE_SCRATCH`, the OS temp dir by default), so later builds render only the new commits.
## [0.10.0] - 2026-09-28
- **The homepage can be built and deployed from `/sites`.** Under a new **Homepage** section, after Hub, there is **Build homepage** (tick **Deploy after build** to ship it in the same job, only if the build succeeds) and **Deploy homepage**, which ships the build already in `homepage/out`. A **Preview branch** box beside them sends either deploy to a Cloudflare Pages preview of the `archilyzer` project instead of production, and shows the preview's address as you type; a name Cloudflare would refuse or rewrite, or `main`, greys the deploy buttons out and says why. A line under the buttons says what a deploy would ship: when `homepage/out` was built (or that it holds no build yet), and where it goes, with the live URL. Deploy homepage with nothing built is refused before any job starts. The homepage reads the search index as it stands, so run **Build index** first when its numbers should move. The jobs run the same code as `archilyzer build homepage` / `deploy homepage`, and show on `/jobs` as `build-homepage`, `deploy-homepage` and `build-deploy-homepage`. The Hub section no longer describes the homepage.
diff --git a/homepage/CHANGELOG.md b/homepage/CHANGELOG.md
@@ -1,6 +1,7 @@
# Homepage Changelog
## [Unreleased]
+- **`/source/` links the source's history.** A History block — how many commits, the newest one (linking to its page), and links to the Log, the Refs and the Atom feed — shows when the build published the history pages (`/source/git/`, rendered by stagit); without them there is no History block. The history pages open on the homepage's ground (the reader's stored choice, else Dark; without JavaScript, the system's), start with one line back to `/source/`, and their Files page is an index into the raw tree. The e2e shows the page with and without a history from a fixture publish (`E2E_SOURCE_PUBLIC_DIR`, never read by a production build), and walks the real pages when the checkout has published them.
- **The growth chart draws its smallest instances together as Other.** Two or more instances that each hold under 5% of the chart's total are one band, **Other**, on top of the stack, in a near-neutral grey of its own (`--chart-other`: 7.36:1 on the Light ground, 3.22:1 on the Dark one, and apart from every instance colour for colour-blind readers); an instance at exactly 5% keeps its band, and a single one under 5% is not grouped. The other instances keep their bands and their colours. The legend lists them and Other; the caption says what Other is and the chart's description names the instances in it; every month's hover title and the Numbers by year table still name every instance. At this release's numbers Hasanalyzer, Rekietalyzer and Jasolyzer are Other. The instance cards and `/stats` are unchanged. The e2e fixture's fifth site transcribes 4 a day rather than 5, so two of its six sites are grouped.
- **An unlisted site is not on the homepage.** A site whose settings turn off **List on the Archilyzer homepage and hub** (`listed: false`) has no Official Instances card, chart series, `/stats` entry or recent item, is not in `channel-sites.json` or `stats/`, and the channels only it carries count in none of the numbers, the headline totals included. The summary's version is 6. The e2e fixture has a seventh, unlisted site that no page names.
- **`/#instances` goes straight to Official Instances.** The section carries `id="instances"`, clear of the sticky header, and every archive's header now links there (`INSTANCES_URL` in `common/lib/project.ts`). With no sites the link lands on the top of the page.
diff --git a/homepage/content/docs/faq.md b/homepage/content/docs/faq.md
@@ -82,8 +82,8 @@ On this site, read-only: `git clone https://archilyzer.pages.dev/source/archilyz
It is the main branch with its whole history, regenerated from the private
repository with every deploy — machine paths scrubbed on the way out, which is
why its commit ids differ. There is no GitHub, and nothing takes a push or a
-pull request. [Source](/source/) also has every file to browse, and
-[Downloads](/downloads/) the same tree as a tarball.
+pull request. [Source](/source/) also has every file to browse and the history
+with every commit's diff, and [Downloads](/downloads/) the same tree as a tarball.
## Can I use it for one video?
diff --git a/homepage/content/docs/install.md b/homepage/content/docs/install.md
@@ -39,7 +39,8 @@ cd archilyzer
`git pull` brings you up to date; nothing takes a push. The commit ids differ
from the private repository's, because machine paths are scrubbed on the way
-out. [Source](/source/) has the details and a browsable copy of every file.
+out. [Source](/source/) has the details, a browsable copy of every file, and the
+history with every commit's diff.
No git? The same tree, without history, is a tarball:
