HEXWORK - A Lantern in the Deep: 3D ASCII terminal roguelike RPG in Rust. Entirely designed and built by Muse Spark 1.3 to evaluate if I like the model.
hexwork docs development-validation.md
6.3 kB
Markdown
at main

Development validation loop #

Status: operator-approved process direction; bounded opt-level 0/1 experiment completed successfully (2026-09-08). Current stable is the practical primary toolchain. Cargo defaults remain unchanged; opt1 is an explicit option for repeated expensive tests. Do not interrupt or change profiles during an active frozen checkpoint gate.

Default tiers #

  1. Implementation feedback: check the affected crates and run the relevant named tests. Reuse a persistent stable target directory for the chosen profile rather than rebuilding dependencies in a new directory for every worker. Separate directories remain appropriate for deliberate compiler/profile comparisons or real isolation needs.
  2. Impact integration: run affected cross-crate/process cases or targets. A unit test does not replace real process/durability evidence. Start with the relevant named process case when possible rather than rerunning every scenario.
  3. Frozen checkpoint: after integration is coherent, run complete target coverage once on frozen source, plus required optimized build/replay and independent review. The existing metadata-derived partition auditor ensures no targets are omitted. Completed audited partitions are the full suite; do not then rerun an equivalent monolithic workspace command locally.
  4. Review fixes: rerun affected checks. Repeat broader gates only when the change can affect them; retain completed evidence on unchanged source. Record exactly which source/version each result covers.

No speedup by ignored regressions, weakened assertions, disabled bounds/overflow/debug checks, fabricated state/history, global mutable test worlds or reduced persistence coverage. CI/platform checks are distinct evidence; do not misrepresent configuration as execution.

Observed baseline and likely cost centers #

The Wield CLI library checkpoint passed235 tests with1 existing ignored in1675.09s (27m55s), using the default unoptimized test profile. An earlier221-test run took1560.96s. A live process observation during the Wield run showed active CPU usage around five cores and about1.6GiB RSS, not an idle deadlock. Rust's over60-second notices are progress warnings, not individual test timings.

Read-only source audit found repeated full-world work in GameSession::from_identity/from_decoded, Story checkpoint proof and replay, canonical scans before Application::save, and process tests intentionally launching independent managed roots/binaries. These are plausible cost centers grounded in call paths, not a completed profiler attribution. Process isolation/durable milestone tests are deliberate coverage, not automatically a fixture-design defect.

Measured opt-level experiment (#57) #

On gameplay baseline 3eb764a, stable rustc/Cargo 1.98.1, x86_64 Linux, three existing tests ran twice per profile, serially with alternating profiles. All 12 exact libtest executions passed; no assertions or fixtures changed. Both DEV and TEST profiles explicitly set opt-level 0/1, debug assertions true and overflow checks true, retaining debuginfo 2. Separate initially absent target directories isolated builds. Direct libtest execution excluded Cargo overhead.

Existing test Opt0 median Opt1 median Speedup
CLI session::tests::exact_run_restore_retains_state_history_and_continuation 23.62s 4.91s 4.81×
phase11_gate binary_new_save_exit_load_preserves_hash_and_history 53.34s 10.00s 5.33×
Save protocol::tests::genuine_pretransfer_v4_fixtures_decode_and_reencode_identically 33.61s 6.31s 5.32×

Build-only totals for CLI lib, phase11_gate and Save lib were 76.33s opt0 versus 184.65s opt1, an additional 108.32s. Later targets reused same-profile dependencies. These are empty-target builds, not OS-cold-cache measurements. Two samples per pair, fixed build order and uncontrolled cache effects limit generalization. Incremental rebuild costs, peak RSS and profiler attribution were not measured; do not extrapolate a full-suite time from these three tests.

Cargo JSON artifacts confirmed optimization level, enabled assertions/overflow checks and debug info for all test executables and the actual non-test hexwork child. Verbose compiler commands confirmed each phase11 harness's compile-time CARGO_BIN_EXE_hexwork selected its corresponding isolated binary; tests/support/mod.rs uses that value directly. The Save test re-encodes the same genuine v4 Classic/Story runs and replay byte-identically in both profiles. Independently allocated run IDs were not compared as world equivalence.

Parent checked the machine-readable ledger's six successful builds and twelve successful samples, recorded timings and compiler/artifact evidence. Local ignored evidence is under target/dev-loop-ab-57/ (RESULTS.md, report.json, runner and complete logs); these local files are not shipped repository artifacts.

Practical decision #

Keep default Cargo profiles unchanged until incremental-edit costs justify a global switch. For repeated expensive test work, opt1 is now a measured option using one persistent target directory. Apply both DEV and TEST settings so process children match, and retain the explicit safety checks. For example:

TMPDIR=/var/tmp CARGO_TARGET_DIR=target/dev-loop-ab-57-opt1 \
CARGO_PROFILE_DEV_OPT_LEVEL=1 CARGO_PROFILE_TEST_OPT_LEVEL=1 \
CARGO_PROFILE_DEV_DEBUG_ASSERTIONS=true CARGO_PROFILE_TEST_DEBUG_ASSERTIONS=true \
CARGO_PROFILE_DEV_OVERFLOW_CHECKS=true CARGO_PROFILE_TEST_OVERFLOW_CHECKS=true \
cargo +stable test --locked -p hexwork-cli --lib \
  session::tests::exact_run_restore_retains_state_history_and_continuation -- --exact

The target name reuses the measured local cache; new checkouts build it once. Do not alternate different flags in the same persistent directory or change a running checkpoint's profile. This option does not replace the focused/impact/frozen-checkpoint tiers above or justify another unchanged full-suite run.

No new general test framework is required. Immutable world-plan reuse may be investigated later only with correct seed/content keys and isolation; shared mutable runtime fixtures are not an acceptable shortcut. No additional legacy-toolchain project is part of this workflow work.