Token efficiency: audit on cheapest capable model, read targeted excerpts not whole files, batch small/leaf modules, structure-only init for big repos; update stays git-incremental

This commit is contained in:
Xingyu Chen
2026-06-11 01:02:52 -07:00
parent 47d9b86c5d
commit b195f28fb3
2 changed files with 43 additions and 9 deletions
+28 -4
View File
@@ -125,6 +125,28 @@ agent — only the convenience changes, never the rules above.
The non-negotiables (independent per-module audit, deterministic scripts, the four-role The non-negotiables (independent per-module audit, deterministic scripts, the four-role
fix gate) hold on every platform; the table only changes *how* you spawn the work. fix gate) hold on every platform; the table only changes *how* you spawn the work.
## Token efficiency
Almost all the cost is the per-module audit sub-tasks reading code — the scripts are
nearly free. Levers, biggest first:
1. **Audit on the cheapest capable model.** The audit is read-code + apply-fixed-rubric +
emit-JSON; a small/fast model does it well, and `apply_audit.py` rejects bad output.
Keep the top model for decomposition, theme synthesis, and fixes only.
2. **Read targeted, not whole.** The audit prompt (STANDARDS.md) greps markers and reads
only the flagged regions; a huge file is scored from its size + a few excerpts, not a
full read. Use `query.py --format paths` so a sub-task opens only its module's files.
3. **Batch the small modules.** Group tiny / low-coupling leaves (≤ ~150 LoC) into one
sub-task that audits each independently (see STANDARDS.md). Core / large / high-coupling
modules stay solo. This cuts the *number* of spawns (each spawn re-pays system-prompt +
rubric overhead). `query.py --max-score 100 --format json` then group by loc/coupling.
4. **`update`, not `init`.** After the first build, only ever run `update` — it re-audits
just the git-changed modules (`needs_audit`), so steady-state cost is tiny.
5. **Structure-first for big repos.** Run `init` in **structure-only** mode (decompose +
`scan --write` + render, *no audits*) to get the map and LoC instantly and cheaply; the
HTML renders unscored modules fine. Then fill scores over time with `update` / on-demand
audits, cheapest-first or worst-suspected-first.
--- ---
## Command: `init` (first build) ## Command: `init` (first build)
@@ -157,12 +179,14 @@ Use when no `modules.json` exists yet. (Also accepts `generate` as an alias.)
yet). Coupling = structural centrality (low/med/high/core); core = the spine hubs. yet). Coupling = structural centrality (low/med/high/core); core = the spine hubs.
2. **Compute size:** `python3 scripts/scan.py --root <proj> --state <state> --write`. 2. **Compute size:** `python3 scripts/scan.py --root <proj> --state <state> --write`.
It reports every module as `unaudited`. It reports every module as `unaudited`.
3. **Audit — one independent subagent per module, in parallel.** For each id in 3. **Audit — one independent sub-task per module, in parallel.** For each id in
`needs_audit`, spawn a subagent with the `reference/STANDARDS.md` prompt (filled with `needs_audit`, spawn a sub-task with the `reference/STANDARDS.md` prompt (filled with
the module's label/paths). Collect each JSON result and apply it: the module's label/paths). Collect each JSON result and apply it:
`python3 scripts/apply_audit.py --state <state> --id <id> --json '<result>' [--rev <git rev>]`. `python3 scripts/apply_audit.py --state <state> --id <id> --json '<result>' [--rev <git rev>]`.
Batch the audits (dozens of modules → many parallel agents, but stay within sane Run on the cheapest capable model, read targeted excerpts, and batch the small/leaf
concurrency; chunk if needed). modules per *Token efficiency* + STANDARDS.md. **Structure-only mode:** for a huge repo
(or a fast/cheap first pass) you may SKIP this step entirely — render the map with no
scores (it renders unscored modules fine), then fill scores later with `update`.
4. **Synthesize `reportThemes`** (47 cross-cutting patterns) from the collected findings 4. **Synthesize `reportThemes`** (47 cross-cutting patterns) from the collected findings
and write them into `modules.json`. and write them into `modules.json`.
5. **Render:** run the render command. Then **stamp the git baseline** so future updates 5. **Render:** run the render command. Then **stamp the git baseline** so future updates
+15 -5
View File
@@ -92,17 +92,27 @@ grep hit as a problem without reading the surrounding code.
## Independent-subagent protocol (REQUIRED) ## Independent-subagent protocol (REQUIRED)
Every module's score MUST be produced by a separate subagent, never inline in the main Every module's score MUST be produced by a separate sub-task, never inline in the main
thread, and never reused across modules. One subagent audits one module against the thread, and never reused across modules. The default is one sub-task per module, run in
paths in its state entry. Spawn them in parallel (Explore or general-purpose). parallel. **Token-saving exception:** several *small, low-risk* modules (≤ ~150 LoC, or
low-coupling leaves) MAY share one sub-task **only if** it audits each independently and
returns a separate, evidence-backed result per module (this is not batch-scoring — it is
several independent audits sharing one context to amortize overhead). Core / high-coupling
/ large modules always get their own sub-task. The audit is a constrained read-and-grade
task, so run these sub-tasks on the **cheapest capable model**; the strict `apply_audit.py`
validation plus this rubric catch weak output. Reserve the top model for decomposition,
theme synthesis, and fixes.
### Subagent prompt template ### Subagent prompt template
> You are auditing CODE QUALITY of ONE functional module for an architecture audit. > You are auditing CODE QUALITY of ONE functional module for an architecture audit.
> Module: **{label}** (`{id}`). Files: {paths}. Project root: {root}. > Module: **{label}** (`{id}`). Files: {paths}. Project root: {root}.
> >
> Read the module's code (grep the markers below, then READ the surrounding code — > Read EFFICIENTLY — do not read whole large files. First grep the smell markers below
> never flag a grep hit you haven't read). Judge it against this rubric: > across {paths}; skim each file's structure (sizes, top-level defs); then READ ONLY the
> flagged regions plus enough context to judge them (never flag a grep hit you haven't
> read). For a big file, the line count + a few representative excerpts are usually enough
> to score bloat/god-component. Judge it against this rubric:
> {paste the "Scoring rubric", "Smell taxonomy", "Severity" sections above} > {paste the "Scoring rubric", "Smell taxonomy", "Severity" sections above}
> >
> Hunt specifically for: monkeypatch / stdlib mutation, fallback chains & silent > Hunt specifically for: monkeypatch / stdlib mutation, fallback chains & silent