Token efficiency: audit on cheapest capable model, read targeted excerpts not whole files, batch small/leaf modules, structure-only init for big repos; update stays git-incremental

This commit is contained in:
Xingyu Chen
2026-06-11 01:02:52 -07:00
parent 47d9b86c5d
commit b195f28fb3
2 changed files with 43 additions and 9 deletions
+28 -4
View File
@@ -125,6 +125,28 @@ agent — only the convenience changes, never the rules above.
The non-negotiables (independent per-module audit, deterministic scripts, the four-role
fix gate) hold on every platform; the table only changes *how* you spawn the work.
## Token efficiency
Almost all the cost is the per-module audit sub-tasks reading code — the scripts are
nearly free. Levers, biggest first:
1. **Audit on the cheapest capable model.** The audit is read-code + apply-fixed-rubric +
emit-JSON; a small/fast model does it well, and `apply_audit.py` rejects bad output.
Keep the top model for decomposition, theme synthesis, and fixes only.
2. **Read targeted, not whole.** The audit prompt (STANDARDS.md) greps markers and reads
only the flagged regions; a huge file is scored from its size + a few excerpts, not a
full read. Use `query.py --format paths` so a sub-task opens only its module's files.
3. **Batch the small modules.** Group tiny / low-coupling leaves (≤ ~150 LoC) into one
sub-task that audits each independently (see STANDARDS.md). Core / large / high-coupling
modules stay solo. This cuts the *number* of spawns (each spawn re-pays system-prompt +
rubric overhead). `query.py --max-score 100 --format json` then group by loc/coupling.
4. **`update`, not `init`.** After the first build, only ever run `update` — it re-audits
just the git-changed modules (`needs_audit`), so steady-state cost is tiny.
5. **Structure-first for big repos.** Run `init` in **structure-only** mode (decompose +
`scan --write` + render, *no audits*) to get the map and LoC instantly and cheaply; the
HTML renders unscored modules fine. Then fill scores over time with `update` / on-demand
audits, cheapest-first or worst-suspected-first.
---
## Command: `init` (first build)
@@ -157,12 +179,14 @@ Use when no `modules.json` exists yet. (Also accepts `generate` as an alias.)
yet). Coupling = structural centrality (low/med/high/core); core = the spine hubs.
2. **Compute size:** `python3 scripts/scan.py --root <proj> --state <state> --write`.
It reports every module as `unaudited`.
3. **Audit — one independent subagent per module, in parallel.** For each id in
`needs_audit`, spawn a subagent with the `reference/STANDARDS.md` prompt (filled with
3. **Audit — one independent sub-task per module, in parallel.** For each id in
`needs_audit`, spawn a sub-task with the `reference/STANDARDS.md` prompt (filled with
the module's label/paths). Collect each JSON result and apply it:
`python3 scripts/apply_audit.py --state <state> --id <id> --json '<result>' [--rev <git rev>]`.
Batch the audits (dozens of modules → many parallel agents, but stay within sane
concurrency; chunk if needed).
Run on the cheapest capable model, read targeted excerpts, and batch the small/leaf
modules per *Token efficiency* + STANDARDS.md. **Structure-only mode:** for a huge repo
(or a fast/cheap first pass) you may SKIP this step entirely — render the map with no
scores (it renders unscored modules fine), then fill scores later with `update`.
4. **Synthesize `reportThemes`** (47 cross-cutting patterns) from the collected findings
and write them into `modules.json`.
5. **Render:** run the render command. Then **stamp the git baseline** so future updates
+15 -5
View File
@@ -92,17 +92,27 @@ grep hit as a problem without reading the surrounding code.
## Independent-subagent protocol (REQUIRED)
Every module's score MUST be produced by a separate subagent, never inline in the main
thread, and never reused across modules. One subagent audits one module against the
paths in its state entry. Spawn them in parallel (Explore or general-purpose).
Every module's score MUST be produced by a separate sub-task, never inline in the main
thread, and never reused across modules. The default is one sub-task per module, run in
parallel. **Token-saving exception:** several *small, low-risk* modules (≤ ~150 LoC, or
low-coupling leaves) MAY share one sub-task **only if** it audits each independently and
returns a separate, evidence-backed result per module (this is not batch-scoring — it is
several independent audits sharing one context to amortize overhead). Core / high-coupling
/ large modules always get their own sub-task. The audit is a constrained read-and-grade
task, so run these sub-tasks on the **cheapest capable model**; the strict `apply_audit.py`
validation plus this rubric catch weak output. Reserve the top model for decomposition,
theme synthesis, and fixes.
### Subagent prompt template
> You are auditing CODE QUALITY of ONE functional module for an architecture audit.
> Module: **{label}** (`{id}`). Files: {paths}. Project root: {root}.
>
> Read the module's code (grep the markers below, then READ the surrounding code —
> never flag a grep hit you haven't read). Judge it against this rubric:
> Read EFFICIENTLY — do not read whole large files. First grep the smell markers below
> across {paths}; skim each file's structure (sizes, top-level defs); then READ ONLY the
> flagged regions plus enough context to judge them (never flag a grep hit you haven't
> read). For a big file, the line count + a few representative excerpts are usually enough
> to score bloat/god-component. Judge it against this rubric:
> {paste the "Scoring rubric", "Smell taxonomy", "Severity" sections above}
>
> Hunt specifically for: monkeypatch / stdlib mutation, fallback chains & silent