bayes for days
sds docs PERFORMANCE.md
10 kB

Performance #

The budget the design is built against, and what actually costs anything. Measured with cargo run --release -p sds-core --example perf, single-threaded, on an i7-1270P under load — deployed runs get one to two CPUs, so a number that needs sixteen cores is not a number.

Measured #

operation cost
threat map, 8 enemies x 544 hexes 13-17 us
160 LOS queries, no cache 109-236 us (0.7-1.5 us each)
1280 LOS queries, per-turn cache 264-715 us (0.2-0.6 us each)
stance blend w = S^T M 0.1 us
score 160 candidates x 30 features 2.4-3.2 us
fire allocation, 48 shots x 8 targets, greedy PMF convolution 162-543 us
one shot's PMF, built cold 0.2 us
parse one 28KB observation 291 us
per-location damage, one volley, cold cache 19 us
per-location damage, one volley, warm cache 0.6 us
20 candidates, per-location 33 us

The 3x spread on fire allocation between runs is contention, not variance in the work. Take the high numbers.

Per-location damage costs 33 us for a whole unit's 20 candidates, against 66 us for the fire allocation beside it. That is affordable, and the reason is the share. Damage arriving at a location is memoised on the shots and on that location's chances in 36, and a pristine front table has only four distinct shares - 7, 5, 4 and 1 - so eight locations cost four convolutions rather than eight, and a second candidate with the same volley costs 0.6 us instead of 19. A whittled-down target has fewer distinct shares still, because the destroyed locations' rolls fold into the ones behind them.

Nothing here enumerates critical slots or recomputes battle value per location. That would be roughly 20 candidates x 8 locations x a battle value pass, and it is what the capped-candidate invariant exists to prevent; what each location holds is one scalar off the observation instead.

Fire allocation measures sds_core::ev, not a stand-in. It is 48 shots against 8 targets but only ~10 distinct (rack, packet damage, to-hit) triples, so the memo answers 4838 of 4848 asks and the convolution is sparse-into-dense: 12 outcomes into 128 buckets, not 128 x 128. Without the memo it is the same work 485 times over.

Per round, 8v8, 16 decisions #

Bot: ~6-10 ms typical, ~30 ms pessimistic. Per unit decision, 1-3 ms.

Dominated today by 16 observation parses (~4.7 ms). That is a statement about how little thinking exists yet, not about JSON being slow: scoring 160 candidates takes 3 us because the scoring is one dot product.

The JVM side is now measured rather than estimated. sds los-dump times MegaMek's own LosEffects.calculateLOS over 2383 calls on four boards and gets 95-107 us mean across runs. That is a cold JVM with no warm-up pass, so treat it as an upper bound - but it is the right order, and it lands at the top of the 20-200 us range this file used to guess. WeaponAttackAction.toHit runs 8 shooters x 6 weapons x 8 targets = 384 of those per firing phase, so ~40 ms/round on the host: still several times the bot's whole budget.

What will dominate once the thinking lands #

Line of sight, and it turned out cheaper than feared. exposure, cover_quality, los_in, los_out, rear_arc_gain and a terrain-aware threat map all need it, and it is O(hexes along the line) per pair: ~1280 queries a round at 8v8. The nine positional features that shipped ask through one Posture::survey per candidate, so the count is two traces per enemy per candidate however many features read the result. This file predicted 5-20 us each and 6-25 ms/round. Measured, one query is 0.7-1.5 us and a whole round is 0.3-0.7 ms — better than a fortieth of the guess even taking the high numbers, and about a tenth of what the host spends answering the same question.

The per-turn LOS cache shared across the side still earns its place: at 8v8 it answers seven queries in eight, which is where 1.5 us falls to 0.6 us. MegaMek does the same internally — the server passes a losCache into filterEntities.

What blows up #

  1. Candidate explosion. Everything here assumes ~20 curated candidates per unit. Enumerating paths the way Princess does is 100-1000x, and puts a "thinking..." message back in front of the player. The cap is the design.
  2. EV over subsets. Greedy with an incremental PMF is O(shots x targets). Any search over allocation subsets is exponential.
  3. toHit scaling as shooters x weapons x targets. A 12v12 of missile boats is ~4x worse — still only ~300 ms/round.
  4. Observation per decision — 28KB x 16 = ~450KB a round encoded and decoded. Sending it once per phase would remove most of that. Tidiness, not performance, until the thinking lands.

Board size is linear and cheap: four sheets is ~34 us for the threat map. Memory is a non-issue — a PMF is 128 floats, the threat map 544, M is 8x30, all L2-resident.

Headroom #

On one CPU with a 100 ms/unit budget we use ~3 ms. Roughly 30x headroom. That is the answer to whether the EV calculator, per-location armour, opponent modelling and multi-turn projection are affordable: yes, provided candidates stay capped.

Concurrency #

The tokio fan-out buys nothing on one core and is not there for speed — a force must have heard from every unit before it decides. It degrades to sequential cleanly and must not be removed as a pointless optimisation.

Where concurrency does pay, even on one CPU, is a compute-once many-waiters LOS cache: eight units will ask overlapping questions. sds_core::los::LosCache is that cache - a Mutex around a memo, with a Condvar per entry so the second asker waits rather than computing the same line again. Nothing iterates the map, so no result depends on its order.

It lives for the match, and deployment shares it. Measured over eight rounds of scenarios/mirror-lance.mms, one seat, with scripts/los-growth.sh:

round phase lines held new MB asked hit%
0 deployment 37,656 37,656 4.74 10,054,462 99.63
1 movement 38,674 1,018 4.87 20,563,140 99.995
2 movement 58,278 19,604 7.34 21,108,220 99.91
4 movement 61,486 710 7.74 15,626,016 99.995
6 movement 61,900 0 7.79 3,213,544 100.00
8 movement 64,020 1,600 8.06 3,416,936 99.95

It flattens: the first two rounds put down 94% of what eight rounds hold, and rounds five to eight add 0.05 MB each. 90.4M lines were asked for and 64,020 computed. There is deliberately no bound on it - a cap chosen before the growth was measured would have been a guess, and 8 MB does not need one.

The same eight rounds with the old per-round clear recompute 20-45k lines every round instead. At the 0.7-1.5 us a line costs that is tens of milliseconds against a round that thinks for ten seconds, so the clear was costing almost nothing: the reason to drop it is that its stated reason - "units move" - was never true of a cache keyed on AttackInfo.

Results must not depend on completion order. Reduce in fixed order, budget in work units rather than wall-clock, give each unit its own seeded RNG stream, and never read a clock inside decision logic.

Comparing against Princess #

The turn clock measures wall time between turn changes, which is the same quantity for both sides and says nothing about how much of the machine each side used to fill it. That gap is not small here, and it runs one way:

threads doing the work
Princess one decision thread, plus one Precognition thread
sds one tokio runtime, units fanned out across its workers

MegaMek contains no parallelStream, no ForkJoinPool and no ExecutorService outside two non-bot classes; megamek/client/bot has exactly one thread of its own, Princess.precognitionThread. BasicPathRanker.rankPath over every legal path is strictly serial. So an unpinned wall-clock comparison between these two is a comparison of two different amounts of hardware, and no sample size fixes it.

So a run whose timings are going to be quoted uses --pin. Each seat gets one physical core and nothing else:

  • Seats are separate processes - see bridge/sds/SdsSeat.java - so one taskset per seat covers its JVM, the per-turn threads MegaMek creates inside it, Precognition, JIT, GC, and the bot subprocess with every tokio worker in it. A mask is inherited across clone and exec, which is what makes one call enough.
  • The host JVM, and so the server and the clock, is started with a mask that excludes the seats' cores, so nothing of MegaMek's own drifts onto one.
  • SDS_WORKER_THREADS=1, because four workers time-slicing one core is context switching and nothing else.
  • Cores are chosen by sds/cores.py, never by CPU number. Two traps it exists for: on an SMT part -c 0 and -c 1 are usually the same physical core, and this is a hybrid part where -c 0 is a 4.8GHz P core and -c 8 a 3.5GHz E one. Seats only ever get distinct physical cores of one speed class, and the SMT siblings of those cores are kept out of the container's cpuset so a seat's core is a whole core.

A pinned figure and an unpinned one are two different measurements. The result document records which it was in pinning, and sds stats prints it beside the thinking-time table.

What the split costs. A match is three JVMs rather than one. Measured over a mirror-lance match: a seat peaks at 668MB resident, the host at 865MB, so ~2.2GB a match against roughly a third of that before. Seats are capped at -Xmx2g because the default maximum heap is a quarter of the machine's RAM and three JVMs each claiming 7.5GB is how --jobs 2 ran a 30GB machine down to 3GB free. Memory is now what bounds --jobs, not CPU.

Match throughput #

~200 s/match at --jobs 3, so ~54 matches/hour; solo is ~88 s, so three at once buys only ~20% — one match already saturates ~2 cores. At ~0.1 core-hours a match, 20,000 matches is ~$24 of Graviton spot. Compute is not the constraint; see plan/training.md.