diff --git a/docs/testing.md b/docs/testing.md index 18b4244..69fe4b6 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -67,6 +67,26 @@ Python, simulator, and container inputs. The current recipe targets `aarch64-linux-gnu`, matching the development host used for its recorded receipt. +## When offline suites cannot reach a failure + +Some failure modes are structurally unreachable in the fixture, and a green +suite says nothing about them. The clearest example: the pinned simulator's +repositories are ~4 KiB, so concurrent workers can never exhaust an in-flight +budget that a single repository fits in. The batch submit/consume deadlock that +ended experiment 4 survived 20 passing suites for exactly that reason, and four +attempts to force it out of the simulator failed. + +A real-network reproduction costs about €0.03. Provision a small cloud box, +stream the image to it, and run against `bsky.network` with the pressure turned +up — for that bug, 50 workers and `--backfill-max-inflight-bytes=268435456` +showed it in 90 seconds, and the before/after was unambiguous. Reach for this +whenever a hypothesis involves memory pressure, worker contention, or repository +sizes the simulator cannot produce. + +`just backfill-batch-contract` pins the weaker property that a dispatch batch +far larger than the job queue converges. Its docstring is explicit that it +passes on the broken code; read it before treating it as coverage. + ## Dashboard and metrics Fully offline: diff --git a/justfile b/justfile index a81c78b..1e74e7e 100644 --- a/justfile +++ b/justfile @@ -414,6 +414,11 @@ powerloss-oracle: # Deterministic, fully offline semantic comparison using the exact pinned # upstream simulator, JSS reader, oracle model, and public Go client. +# batch larger than the job queue must still converge (see the docstring: +# this does NOT reproduce the production deadlock) +backfill-batch-contract: + uv run tests/backfill_batch_contract.py + differential-oracle: zig build -Doptimize=ReleaseSafe python3 tests/differential_oracle.py