From 3eb764a92038db7c88db0abe209601bd552d693d Mon Sep 17 00:00:00 2001 From: "david.yapfest.club" Date: Tue, 8 Sep 2026 14:46:43 -0400 Subject: [PATCH] docs: define focused development and checkpoint validation tiers --- docs/development-validation.md | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) create mode 100644 docs/development-validation.md diff --git a/docs/development-validation.md b/docs/development-validation.md new file mode 100644 index 0000000..1e5b42b --- /dev/null +++ b/docs/development-validation.md @@ -0,0 +1,30 @@ +# Development validation loop + +Status: operator-approved process direction; optimization experiment not yet run. Current stable is the practical primary toolchain. Do not interrupt or change profiles during an active frozen checkpoint gate. + +## Default tiers + +1. **Implementation feedback:** check the affected crates and run the relevant named tests. Reuse a persistent stable target directory for the chosen profile rather than rebuilding dependencies in a new directory for every worker. Separate directories remain appropriate for deliberate compiler/profile comparisons or real isolation needs. +2. **Impact integration:** run affected cross-crate/process cases or targets. A unit test does not replace real process/durability evidence. Start with the relevant named process case when possible rather than rerunning every scenario. +3. **Frozen checkpoint:** after integration is coherent, run complete target coverage once on frozen source, plus required optimized build/replay and independent review. The existing metadata-derived partition auditor ensures no targets are omitted. **Completed audited partitions are the full suite; do not then rerun an equivalent monolithic workspace command locally.** +4. **Review fixes:** rerun affected checks. Repeat broader gates only when the change can affect them; retain completed evidence on unchanged source. Record exactly which source/version each result covers. + +No speedup by ignored regressions, weakened assertions, disabled bounds/overflow/debug checks, fabricated state/history, global mutable test worlds or reduced persistence coverage. CI/platform checks are distinct evidence; do not misrepresent configuration as execution. + +## Observed baseline and likely cost centers + +The Wield CLI library checkpoint passed235 tests with1 existing ignored in1675.09s (27m55s), using the default unoptimized test profile. An earlier221-test run took1560.96s. A live process observation during the Wield run showed active CPU usage around five cores and about1.6GiB RSS, not an idle deadlock. Rust's over60-second notices are progress warnings, not individual test timings. + +Read-only source audit found repeated full-world work in GameSession::from_identity/from_decoded, Story checkpoint proof and replay, canonical scans before Application::save, and process tests intentionally launching independent managed roots/binaries. These are plausible cost centers grounded in call paths, not a completed profiler attribution. Process isolation/durable milestone tests are deliberate coverage, not automatically a fixture-design defect. + +## Next bounded experiment (#57) + +After the active gameplay checkpoint: + +- Select one existing session save/load/continuation test, one real new/save/load process case, and one fixed genuine v4 encode/replay preservation case. Start with a small representative pair; expand only if results justify it. +- Compare identical test inputs/filters serially in separate A/B target directories. A uses CARGO_PROFILE_TEST_OPT_LEVEL=0; B uses1. Both explicitly retain CARGO_PROFILE_TEST_DEBUG_ASSERTIONS=true and CARGO_PROFILE_TEST_OVERFLOW_CHECKS=true. +- Measure build-only preparation separately from repeated warm test runs. Record actual toolchain/profile flags (including the real child binary used by process tests), wall time, test counts/outcomes and max RSS where available. Do not infer child-binary optimization from the requested environment alone. +- Require unchanged canonical/fixture assertions, durable ordering and ignored counts. Fixed genuine artifacts provide a stronger cross-profile byte oracle than comparing independently allocated runs with intentionally different lineage IDs. +- Report cold-build tradeoff and warm-run improvement separately. An optimized release build alone is not evidence that a test profile is faster. If measurements justify adoption, propose the smallest reviewed Cargo/profile change; otherwise keep the existing profile and investigate a measured hotspot. + +No new general test framework is required. Immutable world-plan reuse may be investigated later only with correct seed/content keys and isolation; shared mutable runtime fixtures are not an acceptable shortcut. No additional legacy-toolchain project is part of this workflow work. -- 2.54.0