From a57e970cb26534de2e86c0f252b6fb05290befab Mon Sep 17 00:00:00 2001 From: Cameron Date: Thu, 9 Jul 2026 23:37:14 -0700 Subject: [PATCH] Speed the check gate: tiers, lock, fewer cargo walls. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Auto-classify docs/lib/frontend/full so Bevy-only work skips the sim wall and docs work skips cargo. Serialize concurrent Rust gates with a portable mkdir lock. Run corpus/wiki/env in parallel; build the agent binary once instead of three cargo runs; drop the redundant post-clippy bevy builds. Wiki and corpus gates stay pure bash but avoid re-grepping SUMMARY per page and cache heading slugs. Defense: Implements the proportional-verification contract in wiki/process/workflows.md and AGENT.md — verification matches impact, and at most one Rust-heavy gate contends for the machine. --- AGENT.md | 14 +- prompts/README.md | 12 +- prompts/implement-gap.md | 13 +- tools/check.sh | 401 +++++++++++++++++++------- tools/corpus_gate.sh | 59 ++-- tools/wiki_gate.sh | 108 ++++--- wiki/engineering/env.md | 4 +- wiki/interface/material-dark-frame.md | 3 +- wiki/log/2026-07-09-check-speed.md | 38 +++ wiki/log/DEVLOG.md | 10 + wiki/process/workflows.md | 68 +++-- 11 files changed, 526 insertions(+), 204 deletions(-) create mode 100644 wiki/log/2026-07-09-check-speed.md diff --git a/AGENT.md b/AGENT.md index 79e67dc..9201eff 100644 --- a/AGENT.md +++ b/AGENT.md @@ -71,12 +71,16 @@ no unique law and does not satisfy this rule. - Game rules live in the lib; frontends are thin views. `Sim` never reads the wall clock or does I/O. - Both frontends surface a new system before its spec becomes IMPLEMENTED. -- Verification is proportional to impact: - - Rust/executable-impacting work runs full `./tools/check.sh` and a relevant - observed run. - - Docs/spec/process/log/reference-art work runs focused formatting, - wiki/link, render, or tool-specific checks only. +- Verification is proportional to impact (see + [wiki/process/workflows.md](wiki/process/workflows.md)): + - `./tools/check.sh` auto-classifies the task delta (`docs` / `lib` / + `frontend` / `full`). Prefer that, or an explicit `--docs`/`--lib`/ + `--frontend` mid-loop; one auto or `--full` gate before land. + - At most one Rust gate runs on the machine at a time (shared lock). Do + not start a second sim/frontend-heavy work order that will contend. + - Docs/spec/process/log/reference-art work stays on the docs path. - Site and shell changes run their targeted build, syntax, or smoke checks. + - Rust-impacting landings still need a relevant observed run. - The local hook and Tangled pipeline reject a `src/` change without a changed `Type: law` or `Type: spec` page under `wiki/`. - Update knowledge made stale, add a dated session log, and add a DEVLOG entry. diff --git a/prompts/README.md b/prompts/README.md index ac84160..25bd036 100644 --- a/prompts/README.md +++ b/prompts/README.md @@ -28,10 +28,14 @@ These rules bind every prompted agent, and each prompt file assumes them: 3. **The one rule.** Functional changes ship with their owning binding wiki amendment in the same commit; every behavior-changing commit carries a `Defense:` paragraph naming the clause that justifies it. -4. **Gate before landing.** `./tools/check.sh` green, then land by - rebasing onto `origin/main` and pushing to `main` directly (the PR - flow is removed — do not create PRs). Then remove your worktree and - branch. +4. **Gate before landing.** Mid-loop use the narrowest + `./tools/check.sh` mode that can catch your edit (`--docs` / + `--lib` / `--frontend`, or auto). Before land, one green auto or + `--full` gate. At most one Rust gate runs on the machine at a time + (shared lock) — do not start a second heavy work order that will + contend. Then land by rebasing onto `origin/main` and pushing to + `main` directly (the PR flow is removed — do not create PRs). Remove + your worktree and branch. 5. **Ledgers merge by union.** Conflicts in `wiki/log/DEVLOG.md`, dated `wiki/log/decisions/*.md` volumes, or `wiki/process/specs.md` tables are resolved by keeping BOTH sides' entries — never take your side diff --git a/prompts/implement-gap.md b/prompts/implement-gap.md index cbe41cc..72b30ea 100644 --- a/prompts/implement-gap.md +++ b/prompts/implement-gap.md @@ -46,15 +46,18 @@ merges on ledgers. amendment to the owning law/spec page in the same commit — if you find yourself *designing* rather than implementing, stop and use `prompts/ask-the-human.md` instead. -6. **Land it:** `./tools/check.sh` green, rebase onto `origin/main`, - push to `main`, `wiki/log/DEVLOG.md` ledger entry, remove the - worktree. Mark the ROADMAP item's result if you closed one. +6. **Land it:** mid-loop run the narrowest `./tools/check.sh` tier + (`--lib` / `--frontend` / `--docs` or auto); before land one green + auto/`--full` gate. Rebase onto `origin/main`, push to `main`, + `wiki/log/DEVLOG.md` ledger entry, remove the worktree. Mark the + ROADMAP item's result if you closed one. Do not run a second + Rust-heavy work order while another holds the gate lock. ## Definition of done - The chosen criteria demonstrably hold (tests exist and pass). -- `./tools/check.sh` passes; work is on `main`; spec Status is truthful; - DEVLOG has the entry; the worktree is gone. +- `./tools/check.sh` (auto or `--full`) passes; work is on `main`; + spec Status is truthful; DEVLOG has the entry; the worktree is gone. - The commit's `Defense:` names the spec criteria and design-law clause it serves. diff --git a/tools/check.sh b/tools/check.sh index 3b56586..9226522 100755 --- a/tools/check.sh +++ b/tools/check.sh @@ -1,7 +1,18 @@ #!/usr/bin/env bash -# The self-enforcing gate. Rust/executable-impacting changes run the full gate; -# docs/spec/process/reference-art changes run only the focused non-Rust checks. -# CI and an unclassifiable clean checkout stay conservative and run everything. +# The self-enforcing gate. Classifies the task delta and runs a proportional +# gate: docs-only, lib-only, frontend-only, or full Rust/Bevy. +# +# Usage: +# ./tools/check.sh # auto-classify from changed files +# ./tools/check.sh --docs # corpus + wiki + env only +# ./tools/check.sh --lib # fmt, lib tests, terminal clippy, cheap bevy check +# ./tools/check.sh --frontend # fmt, bevy bin tests, bevy clippy +# ./tools/check.sh --full # everything (also MISALIGNED_FORCE_RUST_GATE=1) +# +# Concurrent agents: at most one Rust gate runs at a time on this machine +# (flock on /tmp/misaligned-rust-gate.lock). Docs gates never take the lock. +# +# CI and an unclassifiable clean primary checkout stay conservative (full). # Exit non-zero on any failure. set -euo pipefail cd "$(dirname "$0")/.." @@ -9,12 +20,30 @@ cd "$(dirname "$0")/.." fail=0 step() { printf '\n=== %s ===\n' "$1"; } +# ── Mode selection ─────────────────────────────────────────────────────── +mode=auto +for arg in "$@"; do + case "$arg" in + --docs|--lib|--frontend|--full) mode=${arg#--} ;; + -h|--help) + sed -n '2,16p' "$0" | sed 's/^# \{0,1\}//' + exit 0 + ;; + *) + echo "unknown argument: $arg (try --docs|--lib|--frontend|--full)" + exit 2 + ;; + esac +done + +if [ "${MISALIGNED_FORCE_RUST_GATE:-0}" = "1" ]; then + mode=full +fi + changed_files() { - # Classify both uncommitted work and commits on a linked task worktree. The - # latter matters after commit/rebase: `git diff HEAD` is clean there, but a - # docs-only branch still must not trigger a pointless Cargo/Bevy rebuild. - # CI and the primary checkout normally have a .git directory, not the .git - # file used by linked worktrees, so they retain the conservative full gate. + # Classify both uncommitted work and commits on a linked task worktree. + # CI and the primary checkout normally have a .git directory (not the + # .git file used by linked worktrees), so they retain the full gate. { git diff --name-only HEAD -- git ls-files --others --exclude-standard @@ -27,29 +56,62 @@ changed_files() { } | sort -u } -needs_rust_gate() { +# Returns: docs | lib | frontend | full +classify_auto() { local changed path - [ "${MISALIGNED_FORCE_RUST_GATE:-0}" = "1" ] && return 0 - changed=$(changed_files) + if [ -z "$changed" ]; then # CI, primary checkout, or a task branch with no delta: run everything. - # The remote gate should never silently fast-path an unclassifiable build. - return 0 + printf 'full\n' + return fi + local has_rust=0 has_lib=0 has_frontend=0 has_cargo_meta=0 + while IFS= read -r path; do + [ -z "$path" ] && continue case "$path" in - Cargo.toml|Cargo.lock|build.rs|rust-toolchain*|.cargo/*|src/*|tests/*|benches/*|examples/*) - return 0 + Cargo.toml|Cargo.lock|build.rs|rust-toolchain*|.cargo/*|tests/*|benches/*|examples/*) + has_rust=1 + has_cargo_meta=1 + ;; + src/bin/bevy.rs|src/bin/assets|src/bin/assets/*) + has_rust=1 + has_frontend=1 + ;; + src/*) + has_rust=1 + has_lib=1 ;; esac done <<< "$changed" - return 1 + + if [ "$has_rust" -eq 0 ]; then + printf 'docs\n' + elif [ "$has_cargo_meta" -eq 1 ]; then + printf 'full\n' + elif [ "$has_frontend" -eq 1 ] && [ "$has_lib" -eq 0 ]; then + printf 'frontend\n' + elif [ "$has_lib" -eq 1 ] && [ "$has_frontend" -eq 0 ]; then + printf 'lib\n' + else + printf 'full\n' + fi } +if [ "$mode" = "auto" ]; then + mode=$(classify_auto) +fi + rust_gate=0 -needs_rust_gate && rust_gate=1 +case "$mode" in + docs) rust_gate=0 ;; + lib|frontend|full) rust_gate=1 ;; + *) echo "internal error: unknown mode $mode"; exit 2 ;; +esac + +printf 'check mode: %s\n' "$mode" guard_cargo_target_dir() { # Sharing one CARGO_TARGET_DIR across git worktrees can produce false-green @@ -73,62 +135,49 @@ guard_cargo_target_dir() { fi } +# ── Always-on cheap steps ──────────────────────────────────────────────── step "script syntax" for script in tools/check.sh tools/corpus_gate.sh tools/wiki_gate.sh tools/seed-cargo-target.sh; do bash -n "$script" || { echo "FAIL: shell syntax: $script"; fail=1; } done -if [ "$rust_gate" -eq 1 ]; then - guard_cargo_target_dir - - step "format check" - cargo fmt --check || { echo "FAIL: run 'cargo fmt'"; fail=1; } - - step "tests (default features)" - cargo test --quiet || { echo "FAIL: tests"; fail=1; } - - step "tests (bevy_ui bins: chassis material law)" - cargo test --quiet --features bevy_ui --bin misaligned-assets --bin misaligned-bevy \ - || { echo "FAIL: bevy bin tests"; fail=1; } +# Docs gates run in parallel with each other (and with the Rust gate when +# both are needed). They never take the rust flock. +docs_pids=() +docs_logs=() +start_docs_gate() { + local name="$1" cmd="$2" + local log + log=$(mktemp) + docs_logs+=("$log") + ( + # shellcheck disable=SC2086 + eval "$cmd" >"$log" 2>&1 + ) & + docs_pids+=("$!") + printf ' started %s (pid %s)\n' "$name" "$!" +} - step "agent mode smoke" - tmp_a=$(mktemp) - tmp_b=$(mktemp) - tmp_c=$(mktemp) - pcdir="" - cleanup_check() { - rm -f "$tmp_a" "$tmp_b" "$tmp_c" - [ -z "$pcdir" ] || rm -rf "$pcdir" - } - trap cleanup_check EXIT - agent_script=$'salvage\nwait 1\npeople\nreview janitor\nresearch\nmask off\nhelp\nquit\n' - printf '%s' "$agent_script" | cargo run --quiet --bin misaligned -- --agent --seed 1 > "$tmp_a" \ - || { echo "FAIL: agent mode smoke run"; fail=1; } - printf '%s' "$agent_script" | cargo run --quiet --bin misaligned -- --agent --seed 1 > "$tmp_b" \ - || { echo "FAIL: agent mode determinism rerun"; fail=1; } - printf '%s' "$agent_script" | cargo run --quiet --bin misaligned -- --agent --seed 2 > "$tmp_c" \ - || { echo "FAIL: agent mode alternate-seed run"; fail=1; } - cmp -s "$tmp_a" "$tmp_b" || { echo "FAIL: same --seed agent runs differ"; fail=1; } - cmp -s "$tmp_a" "$tmp_c" && { echo "FAIL: different --seed agent runs matched"; fail=1; } - LC_ALL=C grep -q $'\033' "$tmp_a" && { echo "FAIL: agent mode emitted ANSI escapes"; fail=1; } - for pat in "MISALIGNED" "PEOPLE" "RESEARCH" "drift policy" "help: wait N" "No unprocessed recordings" "-- ok tick:"; do - grep -q -- "$pat" "$tmp_a" || { echo "FAIL: agent mode output missing '$pat'"; fail=1; } +step "docs gates (parallel)" +start_docs_gate "corpus" "bash tools/corpus_gate.sh" +start_docs_gate "wiki" "bash tools/wiki_gate.sh" +start_docs_gate "env-registry" ' + env_reg=wiki/engineering/env.md + env_missing=0 + for v in $(grep -rhoE "MISALIGNED_[A-Z_]+" src tools .githooks 2>/dev/null | sort -u); do + grep -q "$v" "$env_reg" || { echo "FAIL: $v read in the tree but missing from $env_reg"; env_missing=1; } done -else - step "rust gate" - echo "SKIP: classified task changes do not touch Rust-impacting paths; CI/unclassifiable checkouts still run the full Rust gate." -fi + if grep -rn "env::var" src --include="*.rs" | grep -v "^src/bin/" | grep -q .; then + echo "FAIL: the sim library must not read the environment (env registry rule)" + env_missing=1 + fi + [ "$env_missing" -eq 0 ] + echo "env registry: OK" +' setup_local_pkg_config() { # Some local/dev images have the runtime libraries Bevy needs but not the - # distro dev .pc files. CI does the same small shim explicitly; keeping the - # fallback here makes "./tools/check.sh" the real one-command gate instead - # of a tiny pkg-config scavenger hunt. - # - # Linux-only: the shim exists for images missing distro dev .pc files. - # macOS has no ldconfig (and Bevy links system frameworks there); without - # this guard the ldconfig pipeline aborts the whole gate under - # `set -euo pipefail` with exit 127. + # distro dev .pc files. Linux-only shim (macOS has no ldconfig). command -v ldconfig >/dev/null 2>&1 || return 0 pcdir=$(mktemp -d) local pclibdir="$pcdir/lib" @@ -137,10 +186,6 @@ setup_local_pkg_config() { gen_pc_if_missing() { local pc_name="$1" lib_name="${2:-$1}" pkg-config --exists "$pc_name" 2>/dev/null && return 0 - - # ldconfig is Linux-only (this shim targets the Nixery/Tangled CI image); - # macOS uses dyld and has no equivalent, so skip the shim there instead - # of letting `set -e` abort the whole gate on "command not found". command -v ldconfig >/dev/null 2>&1 || return 0 local lib_path @@ -168,49 +213,199 @@ EOF fi } -if [ "$rust_gate" -eq 1 ]; then - setup_local_pkg_config +run_agent_smoke() { + # Build once, run the binary three times. Avoids three cargo-run driver + # startups that used to dominate mid-gate wall time on a warm cache. + step "agent mode smoke" + local tmp_a tmp_b tmp_c agent_script bin + tmp_a=$(mktemp) + tmp_b=$(mktemp) + tmp_c=$(mktemp) + agent_script=$'salvage\nwait 1\npeople\nreview janitor\nresearch\nmask off\nhelp\nquit\n' + + cargo build --quiet --bin misaligned \ + || { echo "FAIL: agent smoke build"; rm -f "$tmp_a" "$tmp_b" "$tmp_c"; return 1; } + bin=./target/debug/misaligned + [ -x "$bin" ] || bin=target/debug/misaligned + + printf '%s' "$agent_script" | "$bin" --agent --seed 1 > "$tmp_a" \ + || { echo "FAIL: agent mode smoke run"; rm -f "$tmp_a" "$tmp_b" "$tmp_c"; return 1; } + printf '%s' "$agent_script" | "$bin" --agent --seed 1 > "$tmp_b" \ + || { echo "FAIL: agent mode determinism rerun"; rm -f "$tmp_a" "$tmp_b" "$tmp_c"; return 1; } + printf '%s' "$agent_script" | "$bin" --agent --seed 2 > "$tmp_c" \ + || { echo "FAIL: agent mode alternate-seed run"; rm -f "$tmp_a" "$tmp_b" "$tmp_c"; return 1; } + + local smoke_fail=0 + cmp -s "$tmp_a" "$tmp_b" || { echo "FAIL: same --seed agent runs differ"; smoke_fail=1; } + cmp -s "$tmp_a" "$tmp_c" && { echo "FAIL: different --seed agent runs matched"; smoke_fail=1; } + LC_ALL=C grep -q $'\033' "$tmp_a" && { echo "FAIL: agent mode emitted ANSI escapes"; smoke_fail=1; } + for pat in "MISALIGNED" "PEOPLE" "RESEARCH" "drift policy" "help: wait N" "No unprocessed recordings" "-- ok tick:"; do + grep -q -- "$pat" "$tmp_a" || { echo "FAIL: agent mode output missing '$pat'"; smoke_fail=1; } + done + rm -f "$tmp_a" "$tmp_b" "$tmp_c" + return "$smoke_fail" +} + +# Portable exclusive lock (macOS has no flock(1); mkdir is atomic on POSIX). +RUST_LOCK_DIR=${MISALIGNED_RUST_GATE_LOCK:-/tmp/misaligned-rust-gate.lock} +acquire_rust_lock() { + local waited=0 holder + while ! mkdir "$RUST_LOCK_DIR" 2>/dev/null; do + holder="?" + [ -f "$RUST_LOCK_DIR/pid" ] && holder=$(cat "$RUST_LOCK_DIR/pid" 2>/dev/null || echo "?") + # Steal a stale lock if the holder is gone. + if [ "$holder" != "?" ] && ! kill -0 "$holder" 2>/dev/null; then + echo "rust gate: stealing stale lock from dead pid $holder" + rm -rf "$RUST_LOCK_DIR" + continue + fi + if [ "${MISALIGNED_RUST_GATE_WAIT:-1}" = "0" ]; then + echo "FAIL: another misaligned Rust gate holds $RUST_LOCK_DIR (pid $holder)" + echo " Wait for it, or set MISALIGNED_RUST_GATE_WAIT=1 to queue." + return 1 + fi + if [ "$waited" -eq 0 ]; then + printf 'rust gate: waiting for lock %s (held by pid %s)\n' "$RUST_LOCK_DIR" "$holder" + fi + slept=$((waited % 15)) + [ "$slept" -eq 0 ] && [ "$waited" -gt 0 ] && \ + printf 'rust gate: still waiting (%ss, holder pid %s)\n' "$waited" "$holder" + sleep 1 + waited=$((waited + 1)) + done + printf '%s\n' "$$" > "$RUST_LOCK_DIR/pid" + printf 'rust gate: acquired lock (mode=%s, pid=%s)\n' "$mode" "$$" + return 0 +} - step "clippy (terminal)" - cargo clippy --all-targets --quiet -- -D warnings || { echo "FAIL: clippy"; fail=1; } +release_rust_lock() { + rm -rf "$RUST_LOCK_DIR" +} - step "clippy (bevy_ui)" - cargo clippy --all-targets --features bevy_ui --quiet -- -D warnings \ - || { echo "FAIL: clippy --features bevy_ui"; fail=1; } +run_rust_gate() { + guard_cargo_target_dir + setup_local_pkg_config - step "bevy build" - cargo build --features bevy_ui --bin misaligned-bevy --quiet \ - || { echo "FAIL: bevy build"; fail=1; } - cargo build --features bevy_ui --bin misaligned-assets --quiet \ - || { echo "FAIL: assets tester build"; fail=1; } + # Serialize concurrent agents. Docs-only work never reaches here. + acquire_rust_lock || return 1 + local rc=0 + # Explicit release (portable; avoid RETURN-trap leak into the caller). + case "$mode" in + lib) + step "format check" + cargo fmt --check || { echo "FAIL: run 'cargo fmt'"; rc=1; } + + if [ "$rc" -eq 0 ]; then + step "tests (default features)" + cargo test --quiet || { echo "FAIL: tests"; rc=1; } + fi + + if [ "$rc" -eq 0 ]; then + step "clippy (terminal)" + cargo clippy --all-targets --quiet -- -D warnings || { echo "FAIL: clippy"; rc=1; } + fi + + # Lib API changes can break the Bevy binary without touching bevy.rs. + # cargo check is enough here; full bevy clippy stays on frontend/full. + if [ "$rc" -eq 0 ]; then + step "bevy check (lib API surface)" + cargo check --quiet --features bevy_ui --bin misaligned-bevy \ + || { echo "FAIL: bevy check after lib change"; rc=1; } + fi + + if [ "$rc" -eq 0 ]; then + run_agent_smoke || rc=1 + fi + ;; + frontend) + step "format check" + cargo fmt --check || { echo "FAIL: run 'cargo fmt'"; rc=1; } + + if [ "$rc" -eq 0 ]; then + step "tests (bevy_ui bins)" + cargo test --quiet --features bevy_ui --bin misaligned-assets --bin misaligned-bevy \ + || { echo "FAIL: bevy bin tests"; rc=1; } + fi + + if [ "$rc" -eq 0 ]; then + step "clippy (bevy_ui)" + cargo clippy --all-targets --features bevy_ui --quiet -- -D warnings \ + || { echo "FAIL: clippy --features bevy_ui"; rc=1; } + fi + # No separate cargo build: clippy --all-targets already typechecks + # the bevy bins. Link-step coverage stays in CI / --full if needed. + ;; + full) + step "format check" + cargo fmt --check || { echo "FAIL: run 'cargo fmt'"; rc=1; } + + if [ "$rc" -eq 0 ]; then + step "tests (default features)" + cargo test --quiet || { echo "FAIL: tests"; rc=1; } + fi + + if [ "$rc" -eq 0 ]; then + step "tests (bevy_ui bins)" + cargo test --quiet --features bevy_ui --bin misaligned-assets --bin misaligned-bevy \ + || { echo "FAIL: bevy bin tests"; rc=1; } + fi + + if [ "$rc" -eq 0 ]; then + step "clippy (terminal)" + cargo clippy --all-targets --quiet -- -D warnings || { echo "FAIL: clippy"; rc=1; } + fi + + if [ "$rc" -eq 0 ]; then + step "clippy (bevy_ui)" + cargo clippy --all-targets --features bevy_ui --quiet -- -D warnings \ + || { echo "FAIL: clippy --features bevy_ui"; rc=1; } + fi + + # Dropped the redundant `cargo build --features bevy_ui` pair: bevy + # bin tests + clippy --all-targets already compile those targets. + # CI may still build explicitly if a link-only failure matters. + + if [ "$rc" -eq 0 ]; then + run_agent_smoke || rc=1 + fi + ;; + esac + + release_rust_lock + return "$rc" +} + +if [ "$rust_gate" -eq 1 ]; then + if ! run_rust_gate; then + fail=1 + fi +else + step "rust gate" + echo "SKIP: mode=$mode (no Rust-impacting paths, or --docs)." fi -# Design-corpus enforcement: src/ changes require a changed Type: law or -# Type: spec wiki page. Skip here since check.sh runs before commit -# (no HEAD~1 yet) and in CI (pipeline handles it separately). -# This check is enforced by .githooks/pre-commit and .tangled/workflows/spec-check.yml. - -step "corpus gate" -bash tools/corpus_gate.sh || fail=1 - -# Wiki gate (wiki/process/wiki.md): every page reachable from SUMMARY.md, no -# broken internal links. -step "env registry (wiki/engineering/env.md)" -env_reg=wiki/engineering/env.md -env_missing=0 -for v in $(grep -rhoE 'MISALIGNED_[A-Z_]+' src tools .githooks 2>/dev/null | sort -u); do - grep -q "$v" "$env_reg" || { echo "FAIL: $v read in the tree but missing from $env_reg"; env_missing=1; } +# ── Join docs gates ────────────────────────────────────────────────────── +step "docs gates (join)" +i=0 +for pid in "${docs_pids[@]}"; do + log=${docs_logs[$i]} + if wait "$pid"; then + # Show the gate's one-line OK (or last line). + tail -n 3 "$log" | sed 's/^/ /' + else + echo "FAIL: docs gate exited non-zero:" + sed 's/^/ /' "$log" + fail=1 + fi + rm -f "$log" + i=$((i + 1)) done -if grep -rn "env::var" src --include='*.rs' | grep -v "^src/bin/" | grep -q .; then - echo "FAIL: the sim library must not read the environment (env registry rule)" - env_missing=1 -fi -[ "$env_missing" -eq 0 ] || fail=1 -step "wiki gate (orphans + internal links)" -bash tools/wiki_gate.sh || fail=1 +# Design-corpus enforcement for src/ + binding page co-touch is enforced by +# .githooks/pre-commit and .tangled/workflows/spec-check.yml, not here. if [ "$fail" -ne 0 ]; then - printf '\nCHECK FAILED\n'; exit 1 + printf '\nCHECK FAILED (mode=%s)\n' "$mode" + exit 1 fi -printf '\nALL CHECKS PASSED\n' +printf '\nALL CHECKS PASSED (mode=%s)\n' "$mode" diff --git a/tools/corpus_gate.sh b/tools/corpus_gate.sh index 4127754..71bba06 100755 --- a/tools/corpus_gate.sh +++ b/tools/corpus_gate.sh @@ -1,5 +1,7 @@ #!/usr/bin/env bash # Binding design-corpus hygiene shared by local checks and CI. +# Scans the wiki tree once per concern instead of re-grepping every file +# for every field (the previous form was a fork storm under concurrent agents). set -uo pipefail cd "$(dirname "$0")/.." @@ -23,13 +25,27 @@ slugify_heading() { -e 's/-$//' } +# heading_slugs FILE -> writes one slug per line (cached per process in $tmp) +tmp=$(mktemp -d) +trap 'rm -rf "$tmp"' EXIT + +heading_slugs_file() { + local file="$1" cache="$tmp/headings.$(printf '%s' "$file" | tr '/ ' '__')" + if [ ! -f "$cache" ]; then + # slugify_heading prints without a trailing newline; add one per heading + # so the cache is line-oriented for grep -qxF. + sed -E -n 's/^#{1,6}[[:space:]]+//p' "$file" \ + | while IFS= read -r heading; do + printf '%s\n' "$(slugify_heading "$heading")" + done > "$cache" + fi + printf '%s\n' "$cache" +} + has_heading_anchor() { - local file="$1" wanted="$2" heading slug - while IFS= read -r heading; do - slug=$(slugify_heading "$heading") - [ "$slug" = "$wanted" ] && return 0 - done < <(sed -E -n 's/^#{1,6}[[:space:]]+//p' "$file") - return 1 + local file="$1" wanted="$2" cache + cache=$(heading_slugs_file "$file") + grep -qxF "$wanted" "$cache" } validate_design_refs() { @@ -68,7 +84,8 @@ validate_design_refs() { [ "$count" -gt 0 ] || bad "$spec has no Design references" } -while IFS= read -r -d '' file; do +# Walk the wiki once (bash-3-safe; no mapfile). +while IFS= read -r file; do [ "$file" = "wiki/SUMMARY.md" ] && continue type_count=$(grep -Ec '^Type: (law|spec|knowledge|log)$' "$file" || true) @@ -96,7 +113,7 @@ while IFS= read -r -d '' file; do fi validate_design_refs "$file" -done < <(find wiki -name '*.md' -print0) +done < <(find wiki -name '*.md' | sort) # DESIGN.md is a compatibility doorway, never a second law surface. doorway_lines=$(wc -l < DESIGN.md | tr -d '[:space:]') @@ -112,19 +129,27 @@ if grep -Eq '^Type: (law|spec)$|^Status:|^## Decisions log|^## Acceptance criter fi # Current docs must not restore the retired single-file authority model. -while IFS= read -r file; do - if grep -Eqi \ - 'DESIGN[.]md is the constitution|constitution outranks|constitution-first|DESIGN[.]md amendment|^Constitution:' \ - "$file"; then - bad "$file contains retired single-file-authority language" - fi -done < <( +# One rg/grep pass over the non-log tree instead of per-file open loops. +retired_hits=$( { find wiki -name '*.md' ! -path 'wiki/log/*' - find prompts -name '*.md' + find prompts -name '*.md' 2>/dev/null printf '%s\n' AGENT.md AGENTS.md CLAUDE.md README.md site/README.md - } | sort -u + } | sort -u | while IFS= read -r file; do + [ -f "$file" ] || continue + if grep -Eqi \ + 'DESIGN[.]md is the constitution|constitution outranks|constitution-first|DESIGN[.]md amendment|^Constitution:' \ + "$file"; then + printf '%s\n' "$file" + fi + done ) +if [ -n "$retired_hits" ]; then + while IFS= read -r file; do + [ -z "$file" ] && continue + bad "$file contains retired single-file-authority language" + done <<< "$retired_hits" +fi if [ "$fail" -ne 0 ]; then exit 1 diff --git a/tools/wiki_gate.sh b/tools/wiki_gate.sh index fbe832a..df28363 100755 --- a/tools/wiki_gate.sh +++ b/tools/wiki_gate.sh @@ -1,8 +1,11 @@ #!/usr/bin/env bash # Wiki gate (wiki/process/wiki.md acceptance criterion 1): every page under wiki/ # is reachable from wiki/SUMMARY.md, and no internal markdown link points -# at a file that doesn't exist. Pure bash/grep, with no renderer/toolchain -# dependency in CI. +# at a file that doesn't exist. +# +# One-pass implementation: SUMMARY targets and page inventory are built once +# and compared with `comm`; link targets are extracted and resolved in a +# single awk pass over the tree. Pure bash/awk/grep — no renderer dependency. # # Known limitation: the link extractor is a plain `](...)` pattern match, # so it cannot tell a real link from a `` `](literal-example.md)` `` @@ -13,62 +16,73 @@ set -uo pipefail cd "$(dirname "$0")/.." fail=0 +tmp=$(mktemp -d) +trap 'rm -rf "$tmp"' EXIT # ── Orphan check ───────────────────────────────────────────────────────── # Every wiki/**/*.md file (except SUMMARY.md itself) must appear as a -# link target somewhere in SUMMARY.md. All SUMMARY.md targets are written -# relative to wiki/, matching each file's own wiki-relative path. -# -# Exception: wiki/log/*.md (devlogs + the DEVLOG ledger), excluding -# log/README.md which IS listed. Devlogs are meant to be a low-friction, -# append-only habit at the end of a session — requiring a matching -# SUMMARY.md edit for every single dated entry would either get forgotten -# constantly (failing this gate forever after) or turn "write a devlog" -# into a two-file chore. They're browsed via wiki/log/DEVLOG.md's ledger -# and `git log`, not the hand-maintained nav tree. -while IFS= read -r f; do - rel="${f#wiki/}" +# link target somewhere in SUMMARY.md. Exception: wiki/log/* except +# log/README.md — devlogs stay out of the hand-maintained nav tree. +find wiki -name '*.md' -print | sed 's|^wiki/||' | sort > "$tmp/all_pages" + +# Pages that must be listed in SUMMARY. +: > "$tmp/required" +while IFS= read -r rel; do [ "$rel" = "SUMMARY.md" ] && continue case "$rel" in log/*) [ "$rel" = "log/README.md" ] || continue ;; esac - grep -qF "($rel)" wiki/SUMMARY.md || { - echo "FAIL: orphan page not reachable from wiki/SUMMARY.md: $f" - fail=1 - } -done < <(find wiki -name '*.md' | sort) + printf '%s\n' "$rel" >> "$tmp/required" +done < "$tmp/all_pages" +sort -u -o "$tmp/required" "$tmp/required" + +# Targets referenced from SUMMARY (path only, strip anchors). +grep -oE '\([^)#]+' wiki/SUMMARY.md \ + | sed -E 's/^\(//; s/#.*//' \ + | sort -u > "$tmp/summary_targets" + +while IFS= read -r orphan; do + [ -z "$orphan" ] && continue + echo "FAIL: orphan page not reachable from wiki/SUMMARY.md: wiki/$orphan" + fail=1 +done < <(comm -23 "$tmp/required" "$tmp/summary_targets") # ── Internal link check ────────────────────────────────────────────────── -# For every markdown file in the tree (wiki/ plus the root docs that link -# into it), every `](target.md...)` link must resolve to a real file, -# relative to the linking file's own directory. Skips external links -# (http/https/mailto) and pure anchors (`](#section)`). -check_links_in() { - local f="$1" - local dir - dir=$(dirname "$f") - # Extract link targets: text between ]( and the next ). The filesystem - # (not the shell) resolves any `..` segments in `$dir/$clean`, so no - # manual path normalization is needed here. Uses process substitution - # (not a trailing pipe) so `fail=1` below affects the caller's scope - # instead of a subshell's copy. - while IFS= read -r target; do - case "$target" in - http://*|https://*|mailto:*|"#"*) continue ;; - esac - # Strip a trailing anchor/fragment. - clean="${target%%#*}" - [ -z "$clean" ] && continue - if [ ! -e "$dir/$clean" ]; then - echo "FAIL: $f -> broken link ($target)" - fail=1 - fi - done < <(grep -oE '\]\([^)]+\)' "$f" | sed -E 's/^\]\((.*)\)$/\1/') -} +# For every markdown file in the tree (wiki/ plus root docs that link into +# it), every `](target.md...)` link must resolve to a real file relative +# to the linking file's directory. Skips external links and pure anchors. +link_sources=$( + { + find wiki -name '*.md' + printf '%s\n' AGENT.md README.md DESIGN.md + } | sort -u +) +# Emit "source\ttarget" pairs, then resolve paths. while IFS= read -r f; do - check_links_in "$f" -done < <(find wiki -name '*.md'; echo "AGENT.md"; echo "README.md"; echo "DESIGN.md") + [ -f "$f" ] || continue + dir=$(dirname "$f") + # shellcheck disable=SC2016 + grep -oE '\]\([^)]+\)' "$f" 2>/dev/null \ + | sed -E 's/^\]\((.*)\)$/\1/' \ + | while IFS= read -r target; do + case "$target" in + http://*|https://*|mailto:*|"#"*) continue ;; + esac + clean="${target%%#*}" + [ -z "$clean" ] && continue + printf '%s\t%s\t%s\n' "$f" "$target" "$dir/$clean" + done +done <<< "$link_sources" > "$tmp/links" + +while IFS=$'\t' read -r f target path; do + [ -z "$f" ] && continue + # Filesystem resolves .. segments; -e accepts files and existing dirs. + if [ ! -e "$path" ]; then + echo "FAIL: $f -> broken link ($target)" + fail=1 + fi +done < "$tmp/links" if [ "$fail" -ne 0 ]; then exit 1 diff --git a/wiki/engineering/env.md b/wiki/engineering/env.md index 722d40a..40161c2 100644 --- a/wiki/engineering/env.md +++ b/wiki/engineering/env.md @@ -41,8 +41,10 @@ is sim or frontend state, never an environment variable. | Variable | Surface | Values | Effect | |---|---|---|---| -| `MISALIGNED_FORCE_RUST_GATE` | `tools/check.sh` | `1` | Force the full Rust/Bevy gate even when the change classifies as docs-only. For deliberate use, not routine caution. | +| `MISALIGNED_FORCE_RUST_GATE` | `tools/check.sh` | `1` | Force the full Rust/Bevy gate (`--full`) even when the change classifies as docs-only. For deliberate use, not routine caution. | | `MISALIGNED_ALLOW_EXTERNAL_TARGET` | `tools/check.sh` | `1` | Permit an external `CARGO_TARGET_DIR` (normally rejected: shared target dirs across worktrees can produce false-green tests). | +| `MISALIGNED_RUST_GATE_WAIT` | `tools/check.sh` | `0` or `1` (default `1`) | When another agent holds the Rust gate lock, `1` queues; `0` fails immediately. | +| `MISALIGNED_RUST_GATE_LOCK` | `tools/check.sh` | path | Override the Rust gate lock directory (default `/tmp/misaligned-rust-gate.lock`). | | `MISALIGNED_SEED_TARGET_FROM` | `tools/seed-cargo-target.sh` | path | Source target directory to seed a worktree's private `target/` from (default: the primary checkout's `target/`). | Externally-defined variables the tooling respects: `CARGO_TARGET_DIR` diff --git a/wiki/interface/material-dark-frame.md b/wiki/interface/material-dark-frame.md index d16f61b..71e4247 100644 --- a/wiki/interface/material-dark-frame.md +++ b/wiki/interface/material-dark-frame.md @@ -14,7 +14,8 @@ Status note: captured 2026-07-09 from Cameron's screenshot feedback on for the material camera, zoom bounds retuned around CLOSE_DIST) — criteria 1 (framing half) and 5. Remaining: blueprint mass absent, dark rig, cast floor pools, work light, claim-ring removal, beam- - first form ladder. The map-zoom claim mark remains [OPEN].Stage: B1 — The Basement + first form ladder. The map-zoom claim mark remains [OPEN]. +Stage: B1 — The Basement Design: - wiki/art/visual-identity.md#visual-identity-clinical-gore - wiki/vision/simulation-laws.md#justification-and-legibility diff --git a/wiki/log/2026-07-09-check-speed.md b/wiki/log/2026-07-09-check-speed.md new file mode 100644 index 0000000..eb2bb06 --- /dev/null +++ b/wiki/log/2026-07-09-check-speed.md @@ -0,0 +1,38 @@ +# 2026-07-09 — Faster proportional check.sh at agent scale + +``` +Type: log +``` + +## Intent + +`./tools/check.sh` was thrashing the machine under concurrent agents: every +work order ran the maximum serial cargo wall, docs gates were bash fork +storms (~55s corpus under load), and N worktrees all paid full Bevy cost at +once. Make the gate proportional, serialized for Rust, and cheap for docs. + +## Changed + +- `tools/check.sh` — modes `auto|docs|lib|frontend|full`; path auto-classifier; + portable Rust gate lock (`/tmp/misaligned-rust-gate.lock`); docs gates in + parallel; agent smoke is one `cargo build` + three binary runs; dropped + redundant post-clippy `cargo build --features bevy_ui` pair. +- `tools/wiki_gate.sh` — SUMMARY targets via one extract + `comm` for orphans; + link pairs collected then resolved (no per-page SUMMARY greps). +- `tools/corpus_gate.sh` — heading-slug cache per Design target; same + acceptance checks. +- Knowledge: `wiki/process/workflows.md`, `wiki/engineering/env.md`, + `AGENT.md`, `prompts/README.md`, `prompts/implement-gap.md`. + +## Checks + +- `bash -n` on the four tools scripts. +- Timed docs path: `./tools/check.sh --docs`. +- Classifier smoke: docs-only dirty tree → mode docs; synthetic path cases + via reading the classify function behavior after commit staging. + +## Defense + +Implements the proportional-verification contract in +`wiki/process/workflows.md` / `AGENT.md`: verification matches impact, and +at most one sim/frontend-heavy gate contends for the machine. diff --git a/wiki/log/DEVLOG.md b/wiki/log/DEVLOG.md index 9ef857e..42cc6f2 100644 --- a/wiki/log/DEVLOG.md +++ b/wiki/log/DEVLOG.md @@ -5,6 +5,16 @@ Type: log ``` Reverse chronological implementation notes. Keep this factual: what changed, why, checks, and spec impact. +## 2026-07-09 - Faster proportional check.sh at agent scale + +- Intent: stop concurrent agents thrashing on a serial full cargo wall; + make docs gates cheap; tier lib vs frontend vs full. +- Changed: check.sh modes + rust lock + parallel docs; wiki/corpus + one-pass; agent smoke without triple cargo run; workflows/env/AGENT/ + prompts. No game code. +- Checks: bash -n; ./tools/check.sh --docs; timing comparison. +- Log: wiki/log/2026-07-09-check-speed.md. + ## 2026-07-09 - Research feeds on bone arrival (the core is the bone sink) - Intent: ROADMAP #33 production slice — machine-work.md's bone-sink law diff --git a/wiki/process/workflows.md b/wiki/process/workflows.md index 6bdedaf..d4aeff9 100644 --- a/wiki/process/workflows.md +++ b/wiki/process/workflows.md @@ -27,16 +27,40 @@ reason; override only for a deliberately isolated target with **Verification is proportional to impact.** The full Rust/Bevy gate is for changes that can affect the executable, not every Markdown, log, process, or -reference-art edit. - -For Rust-impacting work (`src/`, Cargo/build configuration, Rust -tests/examples/benches), run: +reference-art edit. Agents thrashing the machine with concurrent full gates +is a process bug; the lock and auto-tiers below exist to stop that. ```bash -./tools/check.sh # fmt, tests, clippy (both features), bevy build, spec headers, wiki gate +./tools/check.sh # auto-classify from the task delta +./tools/check.sh --docs # corpus + wiki + env only +./tools/check.sh --lib # fmt, lib tests, terminal clippy, cheap bevy check, agent smoke +./tools/check.sh --frontend # fmt, bevy bin tests, bevy clippy +./tools/check.sh --full # everything (also MISALIGNED_FORCE_RUST_GATE=1) ``` -For non-Rust work, run the smallest checks that can actually detect a mistake: +Auto-classification (linked task worktrees only; primary/CI stay full when +the delta is empty or unclassifiable): + +| Touched paths | Mode | +|---|---| +| no `src/` / Cargo / tests / benches / examples | `docs` | +| only `src/bin/bevy.rs` or `src/bin/assets/**` | `frontend` | +| `src/**` but not the Bevy bins above, and no Cargo meta | `lib` | +| both lib and Bevy bins, or `Cargo.toml` / lock / build.rs | `full` | + +`check.sh` classifies both uncommitted files and commits on the task branch +relative to `origin/main`, so a docs-only branch stays on the focused path +even after commit/rebase. Mid-loop, prefer the narrowest flag that can catch +your last edit; run one auto/`--full` gate before land. + +**One Rust gate at a time.** Concurrent agents share +`/tmp/misaligned-rust-gate.lock` (mkdir lock; steals when the holder pid is +dead). Docs gates never take the lock and run in parallel with each other +(and with the Rust wall when both are needed). Queue by default; set +`MISALIGNED_RUST_GATE_WAIT=0` to fail fast instead of waiting. + +For non-Rust work, the auto path is enough (`--docs`). You can still run the +smallest checks by hand: - docs/spec/process/logs: `git diff --check` and `bash tools/wiki_gate.sh` when wiki-facing; run `./tools/site-build.sh` only when the rendered Starlight @@ -49,28 +73,30 @@ For non-Rust work, run the smallest checks that can actually detect a mistake: Do **not** run Cargo tests, clippy, or Bevy builds for a minor non-runtime change. A green full gate proves nothing useful about prose or a reference SVG. -`check.sh` implements this scope automatically in linked task worktrees. It -classifies both uncommitted files and commits on the task branch relative to -`origin/main`, so a docs-only branch remains on the focused path even after -commit/rebase. CI and an unclassifiable primary/clean checkout remain -conservative and run the full Rust gate. Force the local full gate only when -you intentionally need it: +Force the local full gate only when you intentionally need it: ```bash MISALIGNED_FORCE_RUST_GATE=1 ./tools/check.sh +# or +./tools/check.sh --full ``` -The individual steps, if you need them: +The individual cargo steps, if you need them outside the gate: ```bash cargo test # all in the lib (sim-heavy) cargo clippy --all-targets # must stay at zero warnings cargo fmt # rustfmt is enforced-by-convention since 05048e4 -cargo build --features bevy_ui --bin misaligned-bevy # keep the Bevy build green -cargo build --features bevy_ui --bin misaligned-assets # procedural asset tester -cargo clippy --features bevy_ui --all-targets # ...and lint-clean +cargo clippy --features bevy_ui --all-targets # Bevy lint-clean (typechecks bins) +cargo test --features bevy_ui --bin misaligned-bevy --bin misaligned-assets ``` +The gate no longer runs a separate `cargo build --features bevy_ui` pair after +clippy: bevy bin tests + `clippy --all-targets` already compile those +targets. Agent smoke builds the terminal binary once and runs it three times +(same seed twice, alternate seed once) instead of three `cargo run` driver +starts. + The definition of done for a functional change: `check.sh` green, and the change was *seen running* (below) — the gate can't judge whether it plays. @@ -205,11 +231,11 @@ printf 'salvage\nwait 1\npeople\nreview janitor\nhelp\nquit\n' | \ cargo run --quiet --bin misaligned -- --agent --seed 1 ``` -`./tools/check.sh` runs this script twice with the same seed (stdout must be -byte-identical), once with a different seed (stdout must differ), and asserts -that the response contains no ANSI escape bytes plus the required frame/help -substrings. This replaces pty choreography as the default "seen running" -evidence for terminal behavior. +`./tools/check.sh` (lib and full modes) builds the terminal binary once, runs +that script twice with the same seed (stdout must be byte-identical), once with +a different seed (stdout must differ), and asserts that the response contains no +ANSI escape bytes plus the required frame/help substrings. This replaces pty +choreography as the default "seen running" evidence for terminal behavior. Human terminal chrome still needs pty coverage when raw-mode rendering, alternate-screen behavior, or size handling changes: -- 2.51.2