diff --git a/docs/LANCES.md b/docs/LANCES.md index ae66168..7ba13ed 100644 --- a/docs/LANCES.md +++ b/docs/LANCES.md @@ -132,27 +132,33 @@ findings with different fixes, and one column cannot tell them apart. Two corpora of 20,000 at different seeds, same settings, compared row by row: -| row size (satisfies) | spread between seeds | +| row size (satisfies) | largest disagreement | | --- | --- | -| above 2,000 | within ±1% | -| 700 to 2,000 | up to ±9% | -| 200 to 700 | 10% to 15% | -| under 50 | 10% to 30% | - -That is what a binomial says it should be: a row of *k* lances carries a -relative standard error of about `1/sqrt(k)`, so **±5% needs about 400 lances -in the row** and 20,000 draws buys that down to a satisfies rate of 2%. A row -of thirty is ±20% and should be read as *"this happens"* and not as a rate. - -The **wins** column is noisier than **satisfies** at the same size, because it -is what survives precedence rather than what qualifies: `Fire` moved 48 to 35 -between the two seeds and `Light Fire` moved 5 to 1. Quote a wins figure only +| above 2,000 | 1.7% | +| 700 to 2,000 | 16.4% | +| 200 to 700 | 8.7% | +| under 200 | 28.3% | + +**A binomial underestimates this and should not be used to plan a sample.** A +row of 1,250 would be ±2.8% if every lance were an independent trial, and +`Hammer` moved 1,246 to 1,449. Lances are clustered: a draw fixes a faction and +a year and then takes four correlated designs out of one table, so the +effective sample is a good deal smaller than the lance count. Measure the +spread with a second seed rather than computing it. + +What that leaves usable at 20,000 lances: **rows above 2,000 carry a rate**, +rows between 200 and 2,000 carry an order of magnitude, and rows under 200 say +*"this happens"* and nothing more. + +The **wins** column is worse than **satisfies** at the same size, because it is +what survives precedence rather than what qualifies. `Hammer` moved 3 to 12 +between the two seeds and `Light Battle` 143 to 109. Quote a wins figure only where the satisfies row above it is large. ## What this corpus cannot reach `Order` is the one formation of the thirty-three that no lance satisfied in -60,000 draws across three settings. It wants **four machines of the same +80,000 draws across four settings. It wants **four machines of the same model**, and the generator draws each slot independently from the faction's table, so more samples is not the fix. (Its `faction: DC` gate is not what excludes it: `Formation::qualifies` does not read faction at all, because a @@ -163,3 +169,43 @@ that *satisfies* a named formation, which is the tool for that row and for `Artillery Fire`, `Light Recon` and `Horde` beside it. It is not what this corpus is - a targeted draw answers "what does an Order lance look like" and this answers "what turns up" - and the two should not be mixed in one file. + +## What twenty thousand lances came out as + +`--seed 20260904 --count 20000`, defaults otherwise. Every lance classified as +something, because `Support` admits any four ground Meks and is last in the +order. The rows that matter are the ones above it: + +| formation | satisfies | wins | +| --- | --- | --- | +| Support | 20,000 | 538 | +| Command | 12,504 | 1,786 | +| Security | 7,125 | 4,584 | +| Ranger | 6,967 | 168 | +| Urban | 6,427 | 1,615 | +| Heavy Battle | 5,736 | 25 | +| Anvil | 5,284 | 3,293 | +| Berserker/Close | 5,149 | 1,964 | +| Battle | 5,149 | 0 | +| Assault | 1,195 | 1,195 | +| Heavy Recon | 1,366 | 1,344 | +| Rifle | 781 | 652 | + +Two things in there are worth an argument rather than a note. + +**`Battle` never wins.** Its satisfies count is identical to +`Berserker/Close`'s in every arm measured - 5,149 and 5,149 here, 5,080 and +5,080 at another seed - and `Berserker/Close` is above it in the order, so +`Battle` is classified as `Battle` never. Either the two criteria are the same +criteria, or the order is wrong; the corpus says which pair to look at and does +not say which of the two it is. + +**`Ranger` and `Heavy Battle` are large and almost never win** - 6,967 to 168 +and 5,736 to 25 - where `Assault` and `Heavy Recon` win nearly everything they +qualify for. That is the precedence order doing its job, and it is the number +to have in hand before changing it. + +The two sampling settings are worth less than they look. `--ratings drawn` +moves nothing above its own noise. `--scope any` gives almost the same +distribution and wastes 63% of its draws on faction-year pairs that did not +exist - 54,654 draws for 20,000 lances, against 21,436 for the default.