diff --git a/princess-bench-report.md b/princess-bench-report.md index 99d5dc5..cb2e2bc 100644 --- a/princess-bench-report.md +++ b/princess-bench-report.md @@ -266,6 +266,18 @@ resolving it, so none of our counters saw anything. `illegal`, `rejected` and `failed` are all clean for these matches. A defect of this shape can only ever appear as a stall. +**The scenario contains artillery contrary to intent.** +`4v4-jihad-s4240001-vehicle` fields a `Chaparral Missile Artillery Tank` on both +sides, and jmm's instruction was that the scenarios field no artillery units. It +is the only scenario all night that offered one, and it cost the `vehicles` 4v4 +bucket **4 of its 24 games**. + +**It has deliberately not been changed.** `vehicles` is already measured against +this suite, and altering the suite now would make a future `vehicles` run +non-comparable with the 40.6% above — the suite is fingerprinted for exactly +this reason. Removing or replacing the scenario is a decision to take **between** +benchmark rounds, not inside one. + **The fix, described and not made.** Two options, and they are not the same size: