diff --git a/xbox/ENGINE.md b/xbox/ENGINE.md new file mode 100644 index 0000000000..bd3b1a7660 --- /dev/null +++ b/xbox/ENGINE.md @@ -0,0 +1,111 @@ +# Oskiewar engine: the game/renderer split + +Written 2026-09-26 from four read-only surveys of `xbox/live/oskiewar.js` +(day branch, 21.5k lines) and `xbox/native-bios/` (App.cpp, QuickJsEngine.cpp, +ScenePrimitives.inc). Numbers are Series X `AC_NATIVE_PROFILE hostJsMs` +unless noted. + +## The problem in one line + +On the console the game's JavaScript runs interpreted (QuickJS, no JIT) and +that JavaScript is the whole frame: ~24 ms in the freeskate park, 60 fps needs +<16. Render CPU is 0.7 ms; the GPU is idle. Every frame the JS re-tessellates +~2,100 faces (Bézier limbs, cloth skirts, hair fans, quad meshes, capsule +edges) and hands them over one call at a time. + +The `emitTriangle` comment (batching a Float32Array cost 5.7 ms where per-face +calls cost 1.8) is right and points the wrong way: in QuickJS a typed-array +element write costs ~0.23 µs and a 12-arg C call ~0.85 µs, so **the count of +numbers crossing is the cost, whichever side writes them.** 2,100 faces are +25,000 numbers a frame. A fighter's pose is ~130. Only designs where JS sends +O(objects) and C++ produces the faces get under the 1.8 ms floor. + +## The split + +**Game** (one JS source; deterministic; 331 tests): input, `gameSim`/`netTick`, +rollback (`netSnapshot`/`netRestore`/`netStateHash`), replay capture, the +world-space pose (`buildRunnerWorldGeometry` — it is also the hitbox source, so +it never leaves JS), the LOD tier decision (`figureLod`), the camera solve, and +the order things are drawn in (`gamePaint`). + +**Renderer** (C++ on the console; a JS reference everywhere else): everything +from "world-space pose + camera" down — projection, clipping, capsule/disc/ +ribbon fans, skeleton skinning, outfit, face, terrain strips, quad-mesh +lighting, HUD glyph fanning, post effects. + +The interface is retained handles plus small per-frame payloads: + +```js +// once per map / per look +const mesh = meshUpload(verticesF32, facesF32, capsulesF32) // → int +const style = figureStyle(jsonString) // → int +// per frame +meshDraw(mesh, cameraF32 /* the persistent 27-float buffer */) +figure(style, pose /* the JS pose object; C++ walks it by atom */, state, tier, depth) +postEffects(focusY, band, feather, tiltPx, motionX, motionY) // shipped, 1.0.0.48 +``` + +Rules that keep it one game: +- The renderer returns nothing the game reads. +- `paint` never writes a sim object. Render-only fields (`lodTier`, + `displayFps`, interpolation) live in WeakMaps or a frame-state snapshot, not + on `player` (players ride `structuredClone` into net snapshots). +- Nothing reachable from `gameSim` reads `runtime().monotonicUs`, `cameraDoll`, + `projectPoint` or `viewWidth()` — a wall clock inside the pose would give two + seats different hitboxes. +- Every native call is additive and levelled: native advertises + `capabilities().sceneApi = N`; JS resolves the level once at boot (no + `typeof` on hot paths) and keeps its level-(N−1) path. Level 0 is the JS + reference, which is what the web, the Mac shell and the tests run. +- Native changes bump `Package.appxmanifest` Version every time — the Device + Portal keeps the package it has when the version matches. + +## Where the frame goes (park, five figures) + +| phase | ms | why | +|---|---|---| +| figures (`drawRunner` ×5) | 8 | ~514 native calls per pastel skater; skirt cloth, ponytail, face, curved limbs, bow all tessellated in JS | +| sim | 7.5 | poses built 9–12×/frame (melee, balls, camera, paint, inset); `updateBall` 2.4, `resolveMelee` 1.2 | +| park meshes | 3.4 + 2.1 + 1.5 | meshes are cached but their ~120 edge capsules are re-projected in JS every frame; `sceneMesh` re-validates and heap-allocates per face | +| P1 inset | 4.2 | every park mesh re-sent through `sceneMesh` a second time, unculled (the bounds check reads a field only the web path computes) | +| HUD text | 1.6 | strings rebuilt per frame, DirectWrite layout per string per frame | + +## Migration, in order of ms per day + +| step | lane | work | saves | +|---|---|---|---| +| 0 | JS | cache `runtime()`/`capabilities()` per sim and paint; hoist `typeof` to constants; one pose memo per tick shared by melee, balls, camera, paint and inset (keyed on the sim tick, never the wall clock); one combat-box cache per tick; render fields off `player`; `ac()` at 1 Hz; HUD strings at 10 Hz | 3–4 ms | +| 1 | native | `meshUpload/meshDraw/meshFree`: validate, AABB, pre-light once at upload; capsules folded in and fanned after projection; stack clipping; the inset draws the same handles with its own camera; hall draws retained even with roof shards | ~2.5 ms | +| 2 | native | `figureStyle` + `figure`: tiers 1–3, tier-0 flat body, curved chains, hair cap, outfit minus skirt, trio face, scars, strobe. JS keeps ponytail/skirt/bow/items/shield/glyphs for now. Split `drawRunner` into `figureRecord` → `figureNative` \| `figureJs` | ~1.2–1.4 of 1.6 ms per figure | +| 3 | native | `sceneDraw(camera, viewMask)` for both views; `meshDrawAt` for rigid props (a skateboard = 7 calls, not ~110); `figureInstance` with ponytail/skirt Verlets in C++; crowd batch | ~3.5 + ~2 ms | +| 4 | native | terrain strips by handle for the outdoor maps; native glyph-run cache; `sceneMesh` per-face allocs gone | ~3 ms outdoor | +| 5 | web | ship `scene3d-webgl.mjs` + the reference module so "web = reference" is checked, not hoped; Mac gains `triangles3d`/`figure` via a C shim | 0 console ms | + +Each step is measured the same way: dev-deploy, read `AC_NATIVE_PROFILE` +median over 20 lines, screenshot with `xbox/tools/live.mjs screenshot`. + +## Tests + +- The harness (`createFight`) stubs `triangle3d` and never defines + `capsule3d`/`disc3d`/`figure`/`meshDraw`, so all 120 triangle-observing tests + keep exercising the JS reference unchanged. +- Add scene-level stubs that record `{kind, handle, tier}` and assert semantic + counts (five figures at tier 1, one mesh) — the `countPoseBuilds` pattern. +- Golden cross-check: `quickjs_engine_smoke.cpp` dumps the native triangle + stream for one canonical pose/mesh; a Node test replays the reference and + diffs. This is the test that would have caught `SceneDisc` (5 ring tiers) + already differing from `discRingFor` (6 tiers). +- Source-shape gate: `REQUIRED_SCENE_API` in `oskiewar.js` ≤ the `sceneApi` + literal in App.cpp, so `oskiewar-release.mjs` cannot ship a JS that needs a + level the console does not advertise. + +## Rejected + +- **GPU instancing as the fix.** Building the instance list in JS costs the + same as the calls it replaces. It pays only as a cheaper sink under a C++ + builder, or for shared props. +- **A retained angle skeleton with C++ forward kinematics.** The rig is ~450 + lines of positional, terrain-probing game logic and the hitbox source; a + positional pose is the same wire size as angles. +- **Submitting the inset on alternate frames.** Nothing is retained on the GPU + between frames; it would flicker. diff --git a/xbox/research/research-engine-split.md b/xbox/research/research-engine-split.md new file mode 100644 index 0000000000..822d465db3 --- /dev/null +++ b/xbox/research/research-engine-split.md @@ -0,0 +1,126 @@ +# oskiewar engine split — research + +Read-only survey, 2026-09-26. Paths: `day` = `/Users/jas/ac-worktrees/oskiewar-day-perf`, `perf` = `/Users/jas/ac-worktrees/oskiewar-perf`, `main` = `/Users/jas/aesthetic-computer`. Line numbers are from `day/xbox/live/oskiewar.js` (21,546 lines, `buildVersion = 167`) unless a file is named. + +## 0. What exists today (measured, not assumed) + +**Three diverging copies.** `day` is branch `oskiewar-day-perf` (game side: park, LOD, inset — +4,606/−384 lines over `perf`'s copy). `perf` is `oskiewar-native-on-main` (C++ side: `disc3d`/`capsule3d`/`sceneMesh` at e822dd9e46, post pass at 78f2f08799, package 1.0.0.48 at c2cf815b6d). `main` has *uncommitted* edits to both `xbox/live/oskiewar.js` and `xbox/native-bios/*`. Any plan below lands native first, because the JS feature-detects. + +**Frame on console.** `App.cpp:570-575` calls `sim` then `paint` through `QuickJsPiece::Call` (`QuickJsEngine.cpp:1040-1050`); `hostJsMs` (`App.cpp:578-593`) is the two together, averaged over 120 frames. Draw calls land in `HostGraphics::on_triangle` → `m_frameTriangles` (`App.cpp:371-374`, cap `kMaxTriangles = 8192` at 2753); `DrawGpuTriangles` (`App.cpp:819-856`) maps one vertex buffer and issues one `Draw`. Render CPU is 1.6 ms (084affb4d7). The GPU is idle; the interpreter is the frame. + +**Phase history from commit bodies** (Series X, `AC_NATIVE_PROFILE` + `xbox/tools/oskiewar-phase-profile.py`): + +| build | hostJsMs | what moved | +|---|---|---| +| v142 | 40 (23 fps) | terrain 8.4 ms | +| v143 084affb4d7 | 27 title | terrain projects each point once; HUD rides triangle pass | +| v144 ef651bdbb9 | 18.8 title, ~16 fight | sim 10.5→4.7 ms (`nearRunner` reach cull); terrain 7.2→4.2 | +| park 62ce5e3613 | 36→29.4 | `rebuildSeatInset` 13-14.5→4.1 ms; `updateBall` 4.6→2.4 | +| park f062649517 | 29.4→21.4 | five full figures forced to LOD tier 1 | + +Per figure at full detail: **~90 native calls, ~1.6 ms** (f062649517). Two title bots: 9.5 ms of sim. Those two numbers decide the migration order. + +**Host surfaces, three rungs of one ladder.** + +| host | drawing entry points | source | +|---|---|---| +| web (oskiewar.com) | `triangle` only, Canvas2D (`getContext("2d")` `mac-test.html:228`; `triangle` at 575 batches faces into one path under nonzero winding) | `new Function` injection `mac-test.html:1709-1716`; `capabilities().graphics = "CANVAS2D"` (926-960) | +| macOS native | `triangle`, `triangle3d` → Metal, one `triangleBatch(ptr, count: 1)` per face | `macos-native/main.swift:78-111`, host list 919-969; **no** `triangles3d`/`capsule3d`/`disc3d`/`sceneMesh` | +| Xbox | everything: `triangle3d`, `triangles3d`, `disc3d`, `capsule3d`, `sceneMesh`, `sprites3d`, `texturedTriangles3d`, `themeQuad`, `postEffects` | `QuickJsEngine.cpp:960-1006`, fans in `ScenePrimitives.inc` | + +`xbox/live/scene3d.mjs` + `scene3d-webgl.mjs` (WebGL2, depth-tested, `OSKIEWAR_SCENE_VERSION = 1`, same `(z+1.5)/3` depth map as `App.cpp:830`) exist but are imported only by `bench/` and two tests. The shipping web page does not use them. + +**Game-side feature detection** (all `typeof X === "function"`): `emitTriangle` 2151, `nativeTrianglePass` 2183, `filledDisc` 13591 (`disc3d`), `filledCapsule` 13641 (`capsule3d`), `drawQuadMesh` 21186 and `rebuildSeatInset` 21458 (`sceneMesh`), seat inset 21409/21416 (`triangles3d`), `themeReady` 13713, `comicWrite` 15782. `filledDisc`/`filledCapsule` re-evaluate `typeof` on every call — a global lookup on the hottest path. + +**The `emitTriangle` comment (2145-2150)** — "~2100 faces; Float32Array first cost 5.7 ms where per-face calls cost 1.8 ms; the batched host call measured 0.05 ms." Read as rates: ~0.85 µs per 12-arg C call all-in, ~0.23 µs per typed-array element write. The lesson is not "calls are cheap"; it is **element count is the cost**, whichever side writes it. 2100 faces × 12 = 25k numbers crossing per frame. A pose is ~130 numbers. That ratio is the whole argument for §1. + +**Host-object churn.** `runtime()` has 94 call sites (20 in the 16k-18k fighter region); `RuntimeInfo` (`QuickJsEngine.cpp:699-757`) builds a fresh object with ~40 properties and 3 strings every call. `capabilities()` has 42 sites, at least four inside `gamePaint` per frame (20279-20296); `Capabilities` (`QuickJsEngine.cpp:798-840`) builds ~30 properties with ~8 strings each call. The game reads only `inputFamily, socialPreview, replayOven, reelFullUi, reelHud, country, updateReady, pops, midiPulse, midi, liveAgents` — nothing version-shaped. + +**The pose is simulation state.** `runnerWorldGeometry` (14132) is read by collision: `sampleCombatBoxes` 11025, `meleeLimbContact` 15274/15441/15501, `damagePart` 6957, `frozenGeometry` 9228, `fallenBodyGeometry` 11116. The `renderPoses` cache (14130) only dedups builds inside one `paint` under `sharingRenderPoses` (20840-20849). So the split line is **not** "sim vs paint" — it is *pose in world space* (game) vs *pose to pixels* (renderer). + +**Test harness** (`day/xbox/live/tests/oskiewar.test.mjs`): 331 tests; 120 read `triangles/batches/boxes/lines`; 55 call `fight.paint()`. It stubs `triangle`, `triangle3d`, `triangles3d` (lines 118-139) but **not** `capsule3d`/`disc3d`/`sceneMesh` — so the suite already runs the JS fan fallbacks as the reference. Line 5763 asserts an exact capsule-fan triangle count. `quickjs_engine_smoke.cpp:59-67` exercises the C++ `disc3d`/`capsule3d`/`sceneMesh` but never compares their output to the JS. They already diverge: `SceneDisc` picks n ∈ {8,12,16,20,28} by radius; JS `discRingFor` (13586) has six tiers up to radius 110. + +**Release.** `xbox/tools/oskiewar-release.mjs:29-38` `classifySeverity` → any `xbox/native-bios/` path is `"xbox-native"` and the receipt escalates instead of claiming parity; `stampBuildVersion` 189-215. `appveyor.yml` builds `native-bios` MSIX (`1.0.{build}`) on any `xbox/` change; `test-native-windows.cmd` runs `node --check oskiewar.js`, C++ contract tests, and the QuickJS smoke. `capabilities().version` comes from the literal `m_api->system.version = "1.0.0.46"` at `perf/xbox/native-bios/App.cpp:428` while `Package.appxmanifest:9` says `1.0.0.48` — the version the JS can see is already wrong. + +## 1. The split and the interface + +**Game (one JS source, deterministic, tested):** input → `gameSim`/`netTick` (8128) with `netSnapshot/netRestore/netStateHash` (7604-7680), rollback (`netRollback` 7926-7936), replay capture (`recordReplayCommands` 13254, `captureRoundReplay` 13453 — both inside `gameSim`), pose build (`buildRunnerWorldGeometry` 14413), LOD tier decision (`figureLod` 16978 — a rule about *what* to draw, keep it in JS), camera solve (`containFighters`, `cameraDoll.prepare()` 20365-20368), scene composition order (`gamePaint` 20247+ deciding which things exist this frame). + +**Renderer (native on console, JS reference elsewhere):** everything from "world-space pose + camera" down: projection and clipping (`worldTriangle` 2307, `worldQuad` 17924, `clipPolygon` 2237), capsule/disc fans (`filledCapsule` 13640, `filledDisc` 13590), skeleton skinning (`drawSkeletonSegments` 13859, `drawPaletteCapsule`, outline pass), terrain strip tessellation (the three passes at 20383-20385), quad lighting (`litQuadColor` 17902), mesh draw (`drawQuadMesh` 21184), HUD glyph fanning, post effects. + +**Interface: retained handles + per-frame small payloads, `sceneApi` levelled.** Design rule from §0: minimise numbers crossing, not just calls; never make JS write a big typed array per frame. + +```js +// boot / map change — retained, returns small int handles +const rig = figureRigCreate(segmentRoles, segmentParts); // topology once +const mesh = sceneMeshCreate(verticesF32, facesF32); // already cached as mesh.nativeBuffers (21504) +const strip = terrainStripCreate(profileF32); // terrainDrawProfile, once per map + +// per frame +sceneBegin(cameraBuf, lightX, lightY, lightZ); // cameraBuf = the existing 27-float mainNativeCameraBuffer (21508), persistent +sceneMeshDraw(mesh); // 1 call, 1 int +terrainDraw(strip, nearZ, farZ, colorId); // 1 call per pass instead of a loop of worldQuad +figureDraw(pose, styleId, tier); // 1 call per fighter instead of ~90 +// HUD keeps screenTriangle/hudBox — few calls, already cheap +``` + +**How C++ reads `pose` — the four options, priced against the 2151 comment:** + +| carrier | per-figure cost (≈130 numbers) | verdict | +|---|---|---| +| positional C args (today) | ~90 calls × 0.85 µs ≈ 80 µs + the JS fan work that makes them | keep for HUD; wrong for figures | +| Float32Array JS fills | 130 × 0.23 µs ≈ 30 µs of JS writes + one call | acceptable, but moves work back into the interpreter | +| **plain JS object, C++ walks it** | 0 extra JS work; C++ `JS_GetProperty` with pre-interned atoms (`JS_NewAtom` once in the constructor at `QuickJsEngine.cpp:950`) ≈ 0.1-0.2 µs × 130 ≈ 20 µs | **use for poses**: the object `buildRunnerWorldGeometry` returns is passed as-is; no JS-side copy at all | +| JSON string | `JSON.stringify` is C inside QuickJS but allocates; needs a C++ parser | only for one-time declarations (rig/mesh), never per frame | +| native-owned buffer JS mutates by index | same 0.23 µs/write as above | right for the camera (27 floats) and post effects (6), already the shape of `mainNativeCameraBuffer` | + +`sceneMesh` is the model to copy: retained typed arrays cached per mesh (`nativeMeshBuffers` 21504), a camera buffer rebuilt only when `monotonicUs` changes (`mainNativeCamera` 21508), one call. It just needs the "create → handle" half so the per-frame call carries an int instead of three arrays. + +**The JS reference renderer** is the current fallback code lifted into one module (`xbox/live/scene-reference.mjs`, no new behaviour): `figureDraw` = `drawSkeletonSegments` + `filledDisc` + palette pass on top of `triangle3d`; `terrainDraw` = today's pass loops; `sceneMeshDraw` = the non-native branch of `drawQuadMesh`. Web and Mac call it; the console calls native; both produce the same triangle stream. That makes "the web fallback is the reference" a checked property instead of a hope. The existing `scene3d.mjs` depth constants stay the one shared definition. + +## 2. Keeping the 331 tests meaningful + +Both, layered: + +1. **Reference renderer under the harness (no test edits).** `createFight` keeps stubbing `triangle3d`; the game calls `figureDraw` etc. through the reference module, which emits `triangle3d`. All 120 triangle-observing tests, including the exact fan count at 5763, keep passing because the reference emits the same faces the inline fallback emits today. +2. **Scene-level recorder.** Add `figureDraw`/`sceneMeshDraw`/`terrainDraw` stubs to the harness's `new Function` list that push `{kind, handle, tier}`; new tests assert on scene items (five figures at tier 1, one mesh, three strips) — the `countPoseBuilds` pattern (test 6920-6928), counting semantic events instead of pixels. +3. **Golden cross-check native vs reference.** Extend `quickjs_engine_smoke.cpp` (it already runs `disc3d/capsule3d/sceneMesh` at 62-67) to dump the triangle stream for one canonical pose, mesh and terrain strip to JSON; a Node test replays the same inputs through the reference and diffs. Runs in AppVeyor via `test-native-windows.cmd`. This is the test that would have caught the disc-ring divergence in §0. +4. `renderer.test.mjs` (9 tests: near-plane, guard band, clipping) stays pure JS and becomes the spec for the C++ clipper in `ScenePrimitives.inc` (`SceneClip`). + +## 3. Rollback, replays: what must stay simulation-only + +Good already: `netStateHash` covers players/balls/bullets/grenades/round scalars (7646-7680) and no camera, pose or LOD; `netSimArrays` (7580) restores the live arrays in place; replay capture lives in `gameSim`. + +Hazards the split must not carry across: + +- **In-place render interpolation.** `beginRenderInterpolation` (3148-3170) lerps fields *on the live sim objects* during `paint` and restores in `finally` (20853-20855). After `netRollback` → `netRestore` the `renderPreviousState` map is stale for one frame, and any paint-side write between begin and restore leaks into the hash. Replace with an explicit frame-state (interpolated copy of positions + camera) handed to the renderer; live objects are never touched by `paint`. +- **Paint writes on sim objects.** `player.lodTier` (17001) is written during paint and rides into `structuredClone` snapshots (players are in `netSimArrays`). Move it to a `WeakMap` like `renderPoses`. Same for `lastPaintAt`/`displayFps` (20270-20275) and `figureLodScale`. +- **Wall clock inside the pose.** `runnerWorldGeometry` reads `runtime().monotonicUs` for the strafe step (14148) and that pose feeds `meleeLimbContact`. If `netSimulateFrame` drives sim with a frame-derived `now`, two seats can compute different foot positions → different hitboxes. Verify: pass `now` in, never call `runtime()` from anything reachable from `gameSim`. +- **Camera is render state, replays record it.** `recordReplayCheckpoint` stores `cameraDoll` (3945-3952) and `startReplay` stores `runtime().width/height` (3618-3628) — metadata, fine. What must hold: nothing reachable from `gameSim`/`netSimulateFrame` reads `cameraDoll`, `projectPoint`, `viewWidth()`. Grep those three inside sim-reachable functions before the split; `runnerScreenBounds` is the likely offender. +- **Instant replay stores poses** (`makeRoundReplayFrame` 3996 `geometry: runnerWorldGeometry(...)`, blended by `blendReplayGeometry` 4045). This is fine because the pose stays in JS; it is also why the pose object shape is part of the replay format and must not change when `figureDraw` arrives. + +Rule: the renderer receives (frame-state, poses, camera, handles) and returns nothing the game reads. `sceneMesh` returning `emitted` (`ScenePrimitives.inc` `SceneMesh`) is already a small violation; the game ignores it, keep it that way. + +## 4. Versioned capabilities across two release lanes + +- Native advertises `sceneApi: N` in `Capabilities` (`QuickJsEngine.cpp:798`); fix `system.version` to read `Package::Current->Id->Version` instead of the literal at `App.cpp:428`. Mac's `capabilities` block in `main.swift` advertises its own level (1 today). +- JS resolves once at boot: `const sceneApi = Number(capabilities().sceneApi) || (typeof triangles3d === "function" ? 1 : typeof triangle3d === "function" ? 0.5 : 0)`; every level-N call site keeps its level-(N−1) path; level 0 is the JS reference. Newer JS on an older console degrades to the reference. Newer console on older JS works because the API is additive: never change an argument layout, add `figureDraw2`. No `typeof` in per-call paths — hoist to module constants like `emitTriangle`. +- Release gate: `oskiewar.test.mjs` already reads `App.cpp` as source (line 37, `nativeApp`). Add a source-shape test: `REQUIRED_SCENE_API` in `oskiewar.js` ≤ the `sceneApi` literal in `App.cpp`. `oskiewar-release.mjs` then cannot ship a JS that needs level 2 against a native advertising 1. +- Retail console (`GDK-PORT.md` §3, policy 10.2.5/XR-009) cannot hot-load JS: JS and native ship together in the MSIX. The dev lane (`deploy-xbox-dev`, Device Portal) is where JS leads native by days — that is exactly where the gate above earns its keep. +- Land order for the three copies: `perf` (native) → merge into `main` → `day` (JS) on top, so the console always has the calls before the JS asks for them. + +## 5. End-state and migration order + +**End state.** `oskiewar.js` = rules, sim, net, replay, pose, LOD decision, camera solve, scene composition; it emits handles and small payloads through a levelled `sceneApi`. `scene-reference.mjs` = the JS renderer (web via `scene3d-webgl.mjs`, Mac via Metal, tests via the harness). `ScenePrimitives.inc` + a `figure`/`terrain` unit in C++ = the console renderer, golden-checked against the reference. `App.cpp` keeps owning the frame, D3D11 today, D3D12 on GDK (GDK-PORT §1a) without touching the JS. Each step below is measurable as `AC_NATIVE_PROFILE hostJsMs` on the Series X, and on a desk with `JSC_useJIT=0` under the Mac host (`main.swift:781-790`, which times sim and paint separately — App.cpp does not). + +| step | work | saves (in-fight, Series X) | ms/day | +|---|---|---|---| +| 0 | Cache `runtime()`/`capabilities()` once per `sim` and per `paint`; hoist the six `typeof` checks to module constants; fix `system.version` | ~0.3-0.5 ms + less GC; zero risk | free | +| **1** | **`figureDraw(pose, styleId, tier)`** in C++ reading the pose object by atom; reference = today's `drawSkeletonSegments` path lifted into `scene-reference.mjs`; tests unchanged | ~1.2-1.4 of the 1.6 ms per figure → 2.5-7 ms with 2-5 figures on screen | **best** | +| **2** | `terrainStripCreate/terrainDraw`: retained profile, C++ runs the three passes with `SceneClip` | ~3 of terrain's 4.2 ms | second | +| **3** | Seat inset: kids through `figureDraw` with the inset camera; the 30 Hz `seatInsetBuffer` rebuild goes away | ~3 of 4.1 ms (park only) | third | +| 4 | Frame-state snapshot replaces in-place interpolation; render-only fields off `player` | 0 ms; buys §3 | do with 1 | +| 5 | Web shell → `scene3d-webgl.mjs` + reference; Mac gains `triangles3d` | 0 console ms; makes "web = reference" literal, gives web a depth buffer | after 1-3 | +| 6 | Title bots (9.5 ms of sim): plan every N ticks, cheaper `botScene` | title only | not renderer work | + +Steps 1-3 together take the measured in-fight ~16-24 ms toward the target with the GPU doing what it is idle for now; 0 and 4 are the discipline that keeps the game one deterministic source while it happens. diff --git a/xbox/research/research-figures.md b/xbox/research/research-figures.md new file mode 100644 index 0000000000..f225b6de9b --- /dev/null +++ b/xbox/research/research-figures.md @@ -0,0 +1,222 @@ +# oskiewar figure rendering — research report + +Sources read (read-only): +- JS: `/Users/jas/ac-worktrees/oskiewar-day-perf/xbox/live/oskiewar.js` (21,546 lines; line numbers below are from this file) +- Native: `/Users/jas/ac-worktrees/oskiewar-perf/xbox/native-bios/{QuickJsEngine.cpp,ScenePrimitives.inc,App.cpp}`, `/Users/jas/ac-worktrees/oskiewar-perf/xbox/runtime/include/ac/runtime.hpp` +- Harness: `xbox/live/tests/oskiewar.test.mjs` (`createFight`, lines 93–190) +- Web shell: `xbox/live/mac-test.html` (Caddy rewrites oskiewar.com to it; Canvas2D `triangle` at line 578). Mac shell: `xbox/macos-native/main.swift` (only `triangle3d`, no `capsule3d`/`disc3d`). + +Measurements below come from a scratchpad harness that loads oskiewar.js the way the tests do, stubs `capsule3d`/`disc3d`/`triangle3d` as counters, and times `drawRunner` under V8 (scripts: `scratchpad/count-figure-calls.mjs`, `scratchpad/breakdown-figure.mjs`). V8 is JIT; multiply by ~10 for QuickJS (0.150 ms V8 for a pastel skater vs the 1.6 ms measured on the Series X in commit f062649517). + +## 0. Where the frame goes today + +Native calls per `drawRunner` (one figure, 1080p, in-fight camera, head radius 57 px): + +| figure | tier | capsule3d | disc3d | triangle3d | total calls | V8 ms | +|---|---|---|---|---|---|---| +| P1, no skin (fight) | 0 | 57 | 8 | 117 | 182 | 0.064 | +| P1 pastel skater | 0 | 74 | 45 | 395 | 514 | 0.150 | +| any | 1 | 24 | 4 | 0 | 28 | 0.011–0.016 | +| any | 2 | 12 | 1 | 0 | 13 | 0.005–0.011 | +| any | 3 | 1 | 1 | 0 | 2 | 0.008–0.013 | + +Per-subsystem cost for the pastel skater at tier 0 (calls / V8 µs): + +| pass | function | calls | V8 µs | notes | +|---|---|---|---|---| +| pose build | `buildRunnerWorldGeometry` 14413 | 0 | 1.9 | | +| decorate | `runnerWorldGeometry` 14132 | 0 | +0.1 | breath, strafe, spin, turn, skate rotation | +| IK constrain | `constrainLimbs` 14879 | 0 | +2.4 | fixed bone lengths | +| project | `projectRunnerWorldGeometry` 14914 | 0 | +1.1 | 23 `projectPoint` calls, each allocating | +| ponytail | `drawPonytail` 934 | 40 | 13.6 | 7-node Verlet, 5 passes, Catmull-Rom | +| silhouette | `drawFighterSilhouette` 13930 → `drawCurvedLimbs` 13799 | 136 | 9.5 | 4 cubic chains, 3–7 strokes each, ×2 passes | +| outfit + skirt | `drawOutfit` 1228, `drawSkirt` 1292 | 115 | 18.2 | 6×4 cloth Verlet, 3 passes, capsule separation | +| hair cap | `drawHairline` 1100 | 48 | 3.5 | 19-point fan + 20 edge strokes | +| bow | `drawBow` 1145 | 64 | 7.7 | two 7-node ribbon Verlets | +| face | `drawFace` 16407 → `drawTrioFace` 16305 | 111 | 12.9 | sphere-mapped features, blink, iris | +| scars, inventory | 1411, 16694 | 0 | 0.2 | | + +The no-skin P1 is 152/182 calls of hair: `drawRunner` 17121 and 17130 give **pad 0 the ponytail, hairline and bow even without a skin** (`player.skin || player.pad === 0`). P2 flat is ~30 calls plus palette bands when `handleColors` is set (`drawPaletteCapsule` 13889: 6 bands × 2 tris + 2 discs per segment = 154 `triangle3d`/`disc3d` for 11 segments). + +The pose itself is cheap (~5 µs V8 ≈ 0.05 ms QuickJS). The draw is the cost. Two things dominate: (1) tessellation work done in interpreted JS before every native call (Bezier/Catmull-Rom sampling, fans, cloth), and (2) the sheer count of boundary crossings with 7–12 `JS_ToFloat64` args each. + +The `emitTriangle` note at 2146–2151 is load-bearing for every design below: in QuickJS, twelve typed-array element writes cost more than twelve positional args to one C call (5.7 ms vs 1.8 ms per 2100 faces → ~0.23 µs per element write, ~0.86 µs per 12-arg call). So any design where JS *builds a big buffer* per figure is a loss; JS must send a small buffer (≈100 floats ≈ 25 µs) and C++ must generate the geometry. + +## 1. What defines a drawn figure + +### 1a. Pose (changes every frame) + +World-space skeleton from `buildRunnerWorldGeometry(player, t, at)` 14413–14867: + +``` +{ head: { x, y, z, radius: 22 }, + segments: [ { x1,y1,z1, x2,y2,z2, width (10|11|12), role, part, hitboxOnly? }, ... ] } +``` + +Standard pose is **11 segments + head**: `neck, torso, shoulders, lead-thigh, lead-shin, rear-thigh, rear-shin, left-upper-arm, left-forearm, right-upper-arm, right-forearm`. Roles vary by state (`attack-*`, `item-*`, `grab-*`, `rest-*`, `left-*/right-*`, `lead-*/rear-*`) — 27 role strings appear. `part` ∈ {torso, left-arm, right-arm, left-leg, right-leg} (`limbParts` 10066) and is what `hasPart`/`removedParts` gate on (10069). Alternate builders: `hangGeometry` 14370 (coping stall), `wallPressGeometry` 14090, `spiderDummyWorldGeometry` 14815 (2 body + 40 leg + 8 hitbox-only = 50 segments). + +Inputs the pose reads (all per-frame sim state on `player`): `x,y,z,vx,vy,facing,grounded,ducking,crouchBlend,attackKind/attackUntil/attackStartedAt,itemAction/itemActionUntil,gunAmmo/gunAimX/gunAimY/itemAimLocked,grabHeld,heldBall,ropeIndex,rig{lean,crouch,arms},skateboard/skatePitch/skateRotation/skateGrab/skateStall/trucks{tilt},onewheel,swimming/swimPhase,spin{angle},jumpLaunchAt,landPoseUntil,motionClock,breathPhase,heartDanger,strafe{from,to,at,frames},turnAt,pad,removedParts` plus terrain probes (`surfaceYAt`, `terrainFloorAt`), `meleeTarget`, `itemHandTarget`, `skateFrame`, `fighterAnimationPhase` 14000. This is game logic, entangled with hitboxes (`sampleCombatBoxes` reads the same builder) — it stays in JS in every design. + +`runnerWorldGeometry` 14132 decorates (breath lift, strafe footwork, spin arm opening + Y rotation, civilian `bodyScale`, turn yaw, skate/surf/swim roll) and caches per paint in `renderPoses` (`sharingRenderPoses`). `constrainLimbs` 14879 re-solves every free 2-bone chain to fixed lengths (`boneLength` thigh 36, shin 35, upper-arm 31, forearm 31) — derived. `projectRunnerWorldGeometry` 14914 projects 23 points and multiplies widths by `cameraScale()`. + +Screen geometry consumed by every draw function: + +``` +{ head: { x, y, depth, radius }, + segments: [ { x1,y1,x2,y2, depth, width, role, part, hitboxOnly } ] } +``` + +Depth: the paint loop (20560–20567) sets ONE `triangleDepth` per figure = min(feet depth, depth at `worldNear+2`). Layers offset from it: face −.012, pupils −.02 (`drawFace` 16410/16532), spin arms ±.03/.015 (`drawSpinArms` 21247), held axe via `worldCapsule` −.004 (17875), HUD −1.48. Per-segment `depth` is only used to sort skateboard limbs (13868) and to split spin arms front/back. + +### 1b. Style / identity (changes rarely) + +| field | used by | default | +|---|---|---| +| `color` [r,g,b] | body fill, head (skin) | roster color | +| `handleColors` [[r,g,b]…] | `drawPaletteCapsule` bands (only when no skin, not npc) | [] | +| `skin` `'pastel'`\|null | switches to curved limbs, trio face, outfit, ponytail | null; `dressFreeskater` 5703 sets it; parkKids 21262 | +| `shirtColor, pantsColor, skirtColor, shoeColor, hairColor, accent (ribbon), irisColor` | outfit/hair/face | see 1228–1240, 5701 | +| `emo` | `drawEmoFringe` 21119 (+ dark palette 5711) | false | +| `hairVariant` 0–2 | civilian ponytail (939) | | +| `civilian, bodyScale` | cheap face/skirt/ponytail branches | | +| `dummy` | grey [192,188,179], X eyes; `drawRunner` 17060 clones the player object every frame | | +| `npc, bot` | bot brows, dead-eye X | | +| `name === "@FIFI"` | special hair (16447) | | +| `pad` | pad 0 wears ponytail/hairline/bow without skin (17121, 17130); blink/sway phase seeds | | +| generated appearance (`__oskiewarFighterAppearance`, 13720) | shirt/pants/shoes/skin/hair/beard/glasses | null | + +### 1c. Damage and transient state (changes on events; drawn every frame) + +`removedParts[]`; `partDamage{part:n}` → `damagedPartColor` mix toward red (13710); `scars[]` `{part, role, t, angle, at}` (created lazily in `drawScars` 1411); `hit > 0` → yellow outline; `hitSegment/hitSegmentUntil` → 45 ms strobe capsule (17132); `blocking` + `blockFlash` → bubble shield, 34 capsules + 1 disc (`drawBubble` 17165); `alive`/`headBustedAt` → `drawBrokenRunner` 16903; `resultReaction` (DANCE/POSE/… shifts geometry 17098–17109 and picks mouth/tears/hearts); `frozenGeometry`/`replayGeometry`/`fallenBodyGeometry` (pre-built world poses); held items (`swordHeld, axeHeld, gunAmmo, gunMode, itemAction`) via `drawInventory` 16694 / `drawHeldAxe` 21108; `skateboard/onewheel` via `drawSkateboard` 18712 (world quads, separate object). + +Face inputs (`drawFace` 16407): `facing, alive, hit, blocking, attackKind + meleePulse, resultReaction(+At), spitAt, headRoll, bot/npc, deathCinematic winner/loser, inputPads[pad].leftX/leftY + digital arrows (gaze)`, clock `t` for blink/smile sway; LAUGH and victory draw glyphs through `systemWrite` (D2D text lane, 16563/16589). + +### 1d. Purely derived (recomputed from pose + style + camera; never needs to cross the boundary) + +- `edge = clamp(cameraScale*1.8, 1.25, 3)` outline width (13864 and 6 other places). +- Disc tessellation: JS `discRingFor` 13586 uses 6/8/12/16/24/32 sides at r<6/13/26/52/110; C++ `SceneDisc` uses 8/12/16/20/28 at the same first four thresholds — **console and web already differ triangle-for-triangle**; the parity test at tests 5798 only compares harness hosts. +- Capsule caps: 3/4/6/8/12 half-ring steps at r<6/13/26/52 (JS `capsuleArcs` 13639, C++ `SceneCapsule` identical). +- Curved limbs: cubic Bezier through root/joint/end with clock wobble seeded by role/part string lengths (13815–13830), 3–7 steps by `length/18` (3 for civilians). +- Hair-cap fan (19 points), shoe/sleeve/shirt/daisy placements, skirt mesh topology, all facial feature positions, lash/brow curves — deterministic functions of head radius, facing, clock. +- **Stateful but derived**: ponytail Verlet (`player.hair.nodes`, 7 nodes, link 11, 5 passes), ribbon Verlets (`hair.ribbons`, 2×7 nodes), skirt cloth (`player.skirtCloth`, 6×4 nodes, 3 passes with thigh-capsule separation). These carry velocity across frames; whoever draws them must own that state. + +## 2. Architectures + +Common ground: the native primitives already exist — `SceneDisc`/`SceneCapsule` in `ScenePrimitives.inc` (bound at QuickJsEngine.cpp 969–970), feeding `Graphics::triangle` → `m_frameTriangles` (App.cpp 372, cap `kMaxTriangles = 8192` at 2753) → `DrawGpuTriangles` 819 maps into one dynamic vertex buffer (`GpuTriangleVertex` {x,y,z,r,g,b,a}, NDC = x/960−1, 1−y/540, depth (z+1.5)/3, LESS_EQUAL) and one `Draw`. `SceneMesh` (ScenePrimitives.inc 43–72) already takes a 27-float camera (`mainNativeCamera()` 21508: position, right, up, forward, centerX/Y, orthoScale, focal, perspective, near, viewport, depthBase, depthScale, light) and projects+clips in C++. Native `renderCpuMs` is 1.6 ms for the whole frame; GPU is idle. There is no per-object handle/class precedent in QuickJsEngine.cpp (`CallScope { Api* api; }` only) — a style/instance registry would be new but trivial (a `std::vector` on `CallScope` or a static in the .inc). + +### 2a. Native figure builder (recommended) + +JS API: + +```js +// once per look (on dress / roster apply / parkKids reset), returns int handle +const style = figureStyle(JSON.stringify({ + kind: "flat"|"pastel"|"dummy"|"civilian", + skin:[r,g,b], outline:[8,12,24], shirt:[..], pants:[..]|null, skirt:[..], shoes:[..], + hair:[..], accent:[..], iris:[..], palette:[[..],..]|[], emo:0|1, hairVariant:0..2, + ponytail:0|1, hairCap:0|1, bow:0|1 // pad-0 rule resolved in JS +})); +// per frame per figure +figure(style, pose /*Float32Array*/, state /*Float32Array*/, tier, depth); +``` + +`pose` layout (screen space in phase 1, 4 + 8·N floats, N ≤ 64): + +``` +[0] head.x [1] head.y [2] head.radius [3] segmentCount N +[4+8i+0] x1 [+1] y1 [+2] x2 [+3] y2 [+4] width +[+5] roleId (enum: 0 neck,1 torso,2 shoulders,3 thigh,4 shin,5 upperArm,6 forearm,7 spiderBody,8 spiderLeg,9 other) +[+6] partId (0 torso,1 leftArm,2 rightArm,3 leftLeg,4 rightLeg) +[+7] flags (bit0 hitboxOnly, bit1 chainStart[upper of a 2-bone chain], bit2 free[not attack/item/grab]) +``` + +`state` layout (24 floats): `facing, clockSeconds, hit(0/1), hitSegment(-1..), hitStrobe(0/1), blocking, alive, dead-eyes(0/1), spinAngle, headRoll, blink(0/1), gazeX, gazeY, mouth(enum: smile, grin, ring, line, frown, spit), grin/open amount, damage[5] (per part, 0..3), scarCount, scarsOffset…` — scars are appended as `(partId, roleId, t, angle, heal)` quintuples after the fixed block. + +C++ sketch (new file `xbox/runtime/figure/FigureBuilder.hpp/.cpp`, included by `ScenePrimitives.inc`; bind `figureStyle`/`figure` beside line 970): + +```cpp +struct FigureStyle { uint8_t kind; Color skin, outline, shirt, pants, skirt, shoes, hair, accent, iris; + std::vector palette; bool hasPants, emo, ponytail, hairCap, bow; int hairVariant; }; +struct FigureSink { Graphics& g; float depth; void capsule(...); void disc(...); void tri(...); void stroke(...); }; +void drawFigure(FigureSink&, const FigureStyle&, const float* pose, const float* state, int tier, float edge); +// tier 3: torso capsule + head disc (drawFigureLod 17010) +// tier 2: outline-less capsules, shirt width, head, hair band (17020-17036) +// tier 1: + outline pass, eyes (17024, 17038) +// tier 0: flat kind -> straight capsules ×2 passes, palette bands, head+edge, default face +// pastel kind -> curved chains (Bezier as 13815), hair-cap fan (1100), outfit (1228 minus skirt), +// trio face (16305), scars (1411), hit strobe +``` + +Each `capsule()` in the sink calls the existing `SceneCapsule` math; nothing in App.cpp changes for phase 1. + +What C++ must reproduce to look identical: the tessellation tables (already shared), `edge` rule, curved-limb Bezier with its string-length seeds (13815–13830 — copy the constants; the seed is `part.length*1.7 + role.length*.9`, so pass `seed` as a float in the pose instead of recomputing from strings), hair-cap fan (1112–1141), outfit shoes/sleeves/shirt/daisy (1249–1289), trio face sphere map (16316–16405: `yaw = facing*.42 + spin`, `lon/lat` formulas, blink window `((now/1e6 + pad*.7) % 3.4) < .12`), default face branches (16470–16630 minus glyph branches), scar geometry (1428–1438), hit-strobe cadence (`Math.floor(now/45000) % 2`). + +Hard parts: +- **Cloth and hair sims** — state across frames (`player.hair`, `hair.ribbons`, `player.skirtCloth`). Two options: (i) JS keeps the Verlet and sends node positions (7+14+24 nodes ≈ 100 floats; the sim itself is ~a third of the ponytail's JS cost, so this leaves ~0.15 ms/figure on the table); (ii) `figureInstance(style)` → id with C++-side node arrays, JS sends only the anchors that already exist in the pose (head, torso, thighs) — full win, and the constants (gravity 2200·dt², drag exp(−2.4·dt), 5 passes, keep-out radius `head.radius+3`, skirt `waistHalf 9, drop 38, flare 22`) port directly. Note the Verlets run in **world** units (ponytail nodes are world x/y, projected per point at 1063), so instance state needs the world anchors, not screen — phase 3 sends world poses anyway. +- **Face glyphs** (♪ ♫ ♥ through `systemWrite` 16563, 16589) and `hudCircle` grenade rings — keep in JS as an overlay after `figure()`; they are rare. +- **Items** (`drawInventory`, `drawHeldAxe`, `drawHandgun` 16636, photo-theme sprites) — keep in JS; they are 0–10 calls. +- **Drift**: two implementations of one look. Mitigate with a fixed figure *record* (style JSON + pose + state) that both backends consume, a screenshot diff on the console (`node xbox/tools/live.mjs screenshot`), and forced-tier globals (`__oskiewarFigureLod` 16992 already exists). + +Cost after (a): JS does pose (≈0.05 ms), packs ≈100 floats (≈0.025 ms), one call (~0.005 ms), plus whatever stays in JS. + +### 2b. GPU instancing + +Instancing with a unit capsule/disc mesh and per-instance `{x1,y1,x2,y2,z,halfWidth,rgba}` (or one quad + SDF capsule in the pixel shader — round caps at any radius, antialiased edges, 2 triangles per capsule) would cut `renderCpuMs` (the per-triangle `append` at App.cpp 827–839) and GPU vertex count. Neither is the bottleneck: renderCpuMs is 1.6 ms for everything and the GPU is idle. + +It does **not** help while JS builds the instance list: per the 2146 note, JS filling a 9-float instance record costs ~2 µs in QuickJS — about the same as one `capsule3d` call — so "JS emits instances" is a wash at best. It pays only when: +- C++ owns the capsule list (i.e. after 2a/2c), where an instance buffer is a cheaper sink than tessellating into `m_frameTriangles` — a phase-3 refinement inside `FigureSink`; +- the geometry is shared: parkKids with one style and a few keyframed poses, repeated props (wheels, pickups). But parkKids today are full `players[0]` clones (21262–21284) with their own patrol, breath, heart and Verlets, so they are not instances of one mesh; they are cheap instances of one *builder*. + +Verdict: a GPU-side quality/cleanliness win to fold into the C++ sink later; not a path to 0.1 ms of JS. + +### 2c. Retained skeleton (bone hierarchy + angles, C++ forward kinematics) + +The oskiewar rig is not an angle hierarchy: `buildRunnerWorldGeometry` is ~450 lines of positional, IK-flavoured, terrain-probing game logic (twoBone 14498, footPlant/surfaceYAt 14528, meleeTarget, skateFrame transforms 14746–14800, strafe footwork 14150, spin 14197) and it doubles as the hitbox source for the sim. Moving FK to C++ means either porting that logic (and running it twice — JS for combat, C++ for drawing) or moving combat native, which the web shell cannot follow. The pose costs ~0.05 ms QuickJS; a positional pose (11 segments × 4–7 floats ≈ 50–80 floats) is the same wire size as "a few dozen angles". So (c) buys nothing over (a) with positional bones and costs a second rig. The parts of (c) worth keeping are the *retained* bits — style handle and per-instance sim state — which (a) phase 2 adopts. + +## 3. One description, every shell + +Shells that draw figures today: Xbox UWP (QuickJS + C++ `capsule3d`/`disc3d`/`triangle3d`), macOS native (`main.swift`: only `triangle3d`, so JS tessellates into a Metal batch), the browser (`mac-test.html` Canvas2D `triangle` with path batching; `scene3d-webgl.mjs` exists but is not wired), and the test harness (`triangle3d` stub counting faces, `capsule3d`/`disc3d` absent so JS tessellation runs). + +Proposal: split `drawRunner` into a **figure record** and two **backends**. + +1. `figureRecord(player, geometry, now)` (JS, new, beside `figureLod` 16978) produces `{ style /*handle or object*/, pose: Float32Array, state: Float32Array, tier, depth }` from exactly the fields listed in §1. Pure data; no drawing. Style objects are cached on the player and invalidated by `dressFreeskater`, `applyRoster`, `resetParkKids`, appearance changes. +2. `figureNative(record)` — one call to `figure()`; used when `typeof figure === "function"`. +3. `figureJs(record)` — today's `drawFigureLod`/`drawFighterSilhouette`/`drawOutfit`/`drawHairline`/`drawFace` bodies, refactored to read the record (roleId/partId/state floats) instead of `player` fields, still emitting through `filledCapsule`/`filledDisc`/`screenTriangle`. Web, Mac and tests use this. + +The builder itself is written once in freestanding C++ with a sink interface (`FigureSink` above) and compiled three ways: into QuickJsEngine (Xbox), into the Mac app through a small C shim (adds `figure` to `main.swift` beside `triangle3d` at 802–877), and optionally to WASM for the browser (sink callback → Canvas2D `triangle`), at which point `figureJs` becomes a test-only fallback. Until then `figureJs` is the web path and is fast enough under V8 (0.15 ms/figure). + +Testability: `createFight` (tests 93–190) stubs `triangle3d` and counts faces; it never defines `capsule3d`/`disc3d`/`figure`, so all existing face-count tests (shield 3888, capsule ends 5761, disc faces 5782, per-face parity 5798, LOD behaviour) keep exercising `figureJs` unchanged. Add: (i) a record snapshot test — `figureRecord` for a known state yields known floats (this is what guards native parity, since the record is the contract); (ii) a `triangleHost` variant in `createFight` that stubs `figure` and asserts it is called once per drawn figure with the right tier and never at tier 3 for civilians off-screen; (iii) keep `__oskiewarFigureLod` forcing. Console parity is a screenshot diff via `xbox/tools/live.mjs screenshot` with the tier forced. + +## 4. Recommendation and phases + +Go with **2a with retained state (a+c hybrid)**, staged so each step is measurable with `FRAME_PHASES` (`xbox/tools/oskiewar-phase-profile.py`; the `renderables` phase is the figures) and `AC_NATIVE_PROFILE hostJsMs`. + +**Phase 0 — same day, JS only (no native change), ~15–25% off figures.** +- `drawSkirt` civilian branch (1293–1296) rebuilds and re-projects the whole pose per kid per frame; pass `geometry` in. +- `drawRunner` 17060 spreads a new player object for dummies every frame; `figurePartColor` 16999 runs regexes per segment per frame — resolve `roleId`/`partId` once in the `segment()` closure at 14524 and compare ints. +- `drawBow` is called twice per figure ("tails", "loops"); it re-reads and re-projects the same frame; cache the projected ribbon points on `hair.bow`. +- Add profiler marks inside `drawRunner` for silhouette/face/hair/outfit to get the console's own split before writing C++. +Estimate: pastel 1.6 → ~1.3 ms; flat P1 ~0.6 → ~0.5. + +**Phase 1 — one day, shippable: `figureStyle` + `figure` (screen-space pose), stateless passes only.** +- C++ (`FigureBuilder.cpp` + two bindings at QuickJsEngine.cpp 970): tiers 1–3 complete; tier 0 body (straight capsules ×2 passes, palette bands, curved chains), head + edge, hair-cap fan, outfit minus skirt, scars, hit strobe, default face and trio face (non-glyph branches). +- JS: `figureRecord`, `figureNative`, `figureJs` split; ponytail, bow, skirt, items, shield, glyphs stay as they are and draw after `figure()`. +- Measure with five figures forced to tier 0 (the f062649517 experiment: hostJsMs 29.4 → 21.4 for tier 1). +Estimate per figure: flat P2 0.3–0.6 → ~0.1 ms; flat P1 (hair in JS) ~0.6 → ~0.45; pastel 1.6 → ~0.8 ms. + +**Phase 2 — two to three days: instances and sims in C++.** +- `figureInstance(style)` → id; `figure(instance, pose, state, tier, depth)`; C++ owns ponytail/ribbon/skirt Verlet arrays keyed by instance, stepped on the paint clock exactly as 949–1055, 1163–1191, 1298–1372 (constants copied; keep-out against head and torso capsule; thigh separation). JS passes world anchors, so the pose grows a world block: `head.z` and per-segment `z1,z2` (or switch wholesale to world pose + camera, below). +- Move `drawEmoFringe`, bow loops, shoes. +Estimate: pastel ~0.8 → ~0.2 ms; parkKids (8, civilian branches) ~0.1 ms each. + +**Phase 3 — world-space pose, C++ projection, crowd batch, instanced sink.** +- Pose becomes world space (`x1,y1,z1,x2,y2,z2,width,…`) plus the existing 27-float `mainNativeCamera()` (21508); C++ projects per endpoint (23 fewer allocating `projectPoint` calls in JS) and can give limbs real per-segment depth (fixes the "flat at one depth" compromise at 20560 and the skateboard sort at 13868). +- `figures(buffer, count)` batch for the crowd: one call, records packed back to back; parkKids share a style array. The P1 seat inset (`rebuildSeatInset` 21420, kids rebuilt at 30 Hz into `triangles3d`) can reuse the same builder with the inset's camera. +- Sink option: instanced SDF capsules in App.cpp (`GpuCapsuleInstance {x1,y1,x2,y2,z,w,r,g,b}`, one quad, `DrawInstanced`) so figures stop consuming `kMaxTriangles` and caps are resolution-independent. +Estimate: ~0.08–0.1 ms per figure of JS (pose ~0.05, pack ~0.03, call ~0.005); a crowd of 8 kids ≈ 0.3 ms total; native side stays under the current renderCpuMs. + +**LOD in the design.** Tier stays chosen in JS (`figureLod` 16978 with the frame-time governor `governFigureLod` 16970 — it needs frame time, which JS has) and is passed to `figure()`. C++ applies the same tier semantics (17006–17055) plus tier-0 detail knobs derived from head radius (curve steps, face features skipped when `head.radius < 5`, 16409). Tier 3 for the crowd and any instance whose projected head is off-screen (the paint loop's margin cull at 20559 stays). + +**Crowd.** parkKids remain player clones for behaviour (patrol, startle, heart), but drawing is `figures()` with per-kid instance ids for hair; civilian branches (3-step curves, no bow tails, fixed skirt quad) become `kind: "civilian"` in the style. + +**Risks to name up front.** (1) Two renderers of one look — held together by the record contract and console screenshots. (2) `JsLimits.max_callback_us = 8000` in runtime.hpp is not enforced anywhere in QuickJsEngine.cpp (grep) — no hidden budget. (3) QuickJS heap is 32 MB; style JSON per figure is bytes. (4) Disc ring sides already differ between C++ and JS; document rather than fix. (5) `capsule3d` is absent on the Mac shell — `figureJs` keeps it working until the C shim lands. diff --git a/xbox/research/research-frame-pipeline.md b/xbox/research/research-frame-pipeline.md new file mode 100644 index 0000000000..2b35b71ce0 --- /dev/null +++ b/xbox/research/research-frame-pipeline.md @@ -0,0 +1,158 @@ +# oskiewar console frame pipeline — research + +Sources (read-only): +- JS: `/Users/jas/ac-worktrees/oskiewar-day-perf/xbox/live/oskiewar.js` (21,546 lines) +- Native: `/Users/jas/ac-worktrees/oskiewar-perf/xbox/native-bios/App.cpp`, `QuickJsEngine.cpp`, `ScenePrimitives.inc`, `xbox/runtime/include/ac/runtime.hpp` + +Budget context: `App::Run()` (App.cpp:531-604) measures `hostJsMs` from the top of the loop to after `paint()` — it includes `ProcessEvents`, `PollController`, `PollMidi`, the flush calls, and `sim()`+`paint()`. Every JS draw call lands in `m_frameTriangles` (App.cpp:371-373, cap `kMaxTriangles = 8192` at 2753), and `DrawGpuTriangles` (819-858) copies them into one dynamic VB and issues one `Draw`. Nothing is retained on the GPU between frames; the vertex shader gets screen-space x/y and a depth already computed in JS. There is no model/view transform on the GPU at all. + +--- + +## 1. One frame: call tree, in order + +`paint()` 20833 → `beginRenderInterpolation` 3148 → `gamePaint()` 20247 → `renderPoses.clear()` 20849. +(Wrappers at 21232-21243 cache `capabilities()` per sim/paint call and `runtime()` per paint call. `runtime()` is **not** cached during `sim`; `RuntimeInfo` QuickJsEngine.cpp:699 builds a ~20-property object per call.) + +| # | Phase (line) | What it emits per frame (freeskate park, 5 figures) | Nature | +|---|---|---|---| +| 0 | prologue 20248-20345 | `governFigureLod`, `syncGameView`, `panPlayer`×2 (2 `projectPoint`), `acFeed = ac()` (native builds a fighter-array object every frame, QuickJsEngine.cpp:847-870), `fighterProfile(name)` per player, `refreshPhotoTheme` (native `themeReady()`), `displayTheme()`→`losAngelesSun()`, ~20 `mixColor` array allocs | recompute of constants | +| 1 | `updateSceneLighting` 18576 | 2-4 `projectPoint` + native `themeLighting` (only when photo theme) | per-frame | +| 2 | `wipe`, `drawSkyAtmosphere` 18418 | returns (not survival) | — | +| 3 | `containFighters` 14993 | camera fit; `runnerWorldGeometry` for `cameraPlayers()` → **first pose build of the frame** for players | per-object | +| 4 | `cameraDoll.prepare` 2073; `terrainSpan` 8785 | — | — | +| 5 | `drawRoomSurfaces` 18215 → `drawGridOverlay(…, false)` 18322 | loops **every** `gridField` cell each frame (gridCols×gridRows), draws only hot cells; + 4 `worldLine` (D2D lines) | static-ish; always iterates | +| 6 | `drawTerrainBackWall` 18196 | indoor → returns | — | +| 7 | `drawTerrainSurface` 18129 | cached `pipeSurfaceMesh` → `drawQuadMesh` → **1 `sceneMesh`** | static per map, cached | +| 8 | `drawTerrainFrontWall` 18153 | cached `pipeWallMesh` → **1 `sceneMesh`** | static per map, cached | +| 9 | `drawDecals` 833 | ≤45 decals: arc = 4-8 `worldQuad` with `{...p1}` spreads; skid = 2 `worldQuad`; others 3 `projectPoint` + fan of `screenTriangle`; blood drops 2 `projectPoint` + 1 tri each | slowly-changing, recomputed | +| 10 | `drawSkateParkFeatures` 1445 → `drawIndoorHall` 755 | `drawCityStreet` 21366: ~4-8 cached city blocks → **4-8 `sceneMesh`**; `hallMesh` → **1 `sceneMesh`** + replay of `mesh.capsules` through JS `worldCapsule` → **~56 `capsule3d`** (6 cubes × 9 + 2 coping); `drawDust` 731: 28 `projectPoint` + **56 `triangle3d`**; `drawLoopTrack` per loop (cached mesh → `sceneMesh`) | static per map; mesh cached, capsules + dust recomputed | +| 11 | ledges loop 20399-20412, `drawBoosterPad` 18405 | `worldQuad` + `worldLine` per visible ledge; 1 `worldQuad` per booster | static, recomputed | +| 12 | shadows 20430-20440 | `drawSpotShadow` 18600 per active player (not kids): 3 `projectPoint` + **14 `triangle3d`**; balls likewise | per-object | +| 13 | HUD top block 20447-20510 | `hudSafeRect`, optional timer `typeWrite`; **`drawHudStatusTray` 19744** → `drawParkSupply` 20936 (axe pickup → `drawBigAxe`), **`drawUnderPipe` 20981** (cached `waterMesh` → 1 `sceneMesh` + replay of ~60+ `worldCapsule` = one per pond + one per intact glass pane, `bottomGlass.length = 60`), `drawWaterSplashes` 21046 → `drawTurboParticles` (≤48 × 2 `disc3d`); freeskate `seatHudText`×2 → 4 `comicWrite` | mixed | +| 14 | pickups 20512-20517 | `drawGunPickup`/`drawSaberPickup`/… per active pickup | per-object | +| 15 | renderables 20520-20549 | 5 array spreads, `parkKids` culled with `projectPoint`, `.sort` with closure `depth()` allocating a vector per compare; per renderable 2-3 `projectPoint` (each allocates 2 objects: `toView` + `projectView` default `out={}`) | per-frame alloc churn | +| 16 | `drawRunner` 17058 ×5 | see breakdown below (~430 `triangle3d`, ~100 `capsule3d`, ~48 `disc3d` per LOD-0 figure) | per-object, pose-driven | +| 17 | debug lines 20573-20580 | all early-return unless `debugHitboxes` | — | +| 18 | `drawFrameMeter` 17321, `drawImpacts` 20112, `drawTitleHeadDoor` 20177 | guard returns / few tris | — | +| 19 | `drawControlLegend` 16110 | returns in freeskate | — | +| 20 | `drawFreeskateSpeed` 5264 → `drawSeatPlayerHud` 21137 | **`drawSeatFirstPerson` 21401** (see below) + per player 2-4 `seatHudText` = **4-8 `comicWrite`** + `drawSeatHeart` | screen-space | +| 21 | hudPlayers loop 20686-20697 | `drawPlayerHandle`/`drawHudInventory` return in freeskate; `drawCommandStream` → `drawSeatAction` 21148: 1 `projectPoint` + 2 `comicWrite` when a move word is fresh | screen-space | +| 22 | `drawDebugPerformance` 19838, `drawDeathFlash`, `drawVersusHud`, `drawSpectatorQr`, `drawNetHealth`, `drawTouchControls` | all return or near-nothing in freeskate | — | + +### `drawRunner(player)` 17058 — per LOD-0 figure (players[0] has `skin:'pastel'` 5706; kids too 21272) + +| Step (line) | Work | Native calls | +|---|---|---| +| `runnerGeometry` 14869 → `runnerWorldGeometry` 14132 | cache hit for players (built in `containFighters`), **fresh build for each kid**; then `constrainLimbs` 14880 (copies every segment with `{...segment}`, `segments.find` with regex `kind()` per upper bone), `projectRunnerWorldGeometry` 14914 (2 `projectPoint` per segment + head ≈ **29 `projectPoint` ≈ 58 object allocs**) | — | +| `buildRunnerWorldGeometry` 14413 | `fighterAnimationPhase` 13981 (string state machine, `meleeFrames[state]`, 2 `terrainFloorAt`), ~14 `segment()` pushes each running `partForRole` (8 `startsWith`/`includes` string tests) + `hasPart` (`removedParts.includes`), `twoBone` IK, skateboard transform loop with `segments.find` ×2 per leg | — | +| `figureLod` 16978 | scan | — | +| `drawSkateboard` 18712 | 7 stations × 4 `worldQuad` + 2 caps = 30 `worldQuad` (each: `litQuadColor` alloc, 4 `toView`, 4 `projectView`, 2 `projectedTriangle`); cracks `worldCapsule`; `drawBoardRunningGear` 18682: 2 × `drawTruck` (10 `worldCapsule` each, 3 when small) + 4 wheels × (1 `projectPoint` + 4 `filledDisc` + 2 `screenTriangle`) | **~60 `triangle3d` + ~20 `capsule3d` + 16 `disc3d` + 8 `triangle3d`** | +| `drawPonytail` 934 | player: 7-node Verlet × 5 passes, Catmull-Rom 13 pts, 13 `projectPoint`, 2 × 12 `filledCapsule`, 8 `stroke`; civilian (kids): 3 `projectPoint` + 4 `stroke` | **24 `capsule3d` + 16 `triangle3d`** | +| `drawBow(…,"tails")` 1145 | 2 ribbons × 7-node Verlet, 14 `projectPoint`, 24 `stroke` + 2 tris | **~50 `triangle3d`** | +| `drawFighterSilhouette` 13930 → `drawSkeletonSegments` 13859 → **`drawCurvedLimbs` 13799** | chains via `segments.find` + regex; per limb cubic Bézier 4-8 points (`points.push({x,y})`), `drawChain` = 1 `stroke` per step × 2 passes + 2 `filledDisc`; straight bones 2 `filledCapsule` each; head 2 `filledDisc` | **~112 `triangle3d` + ~16 `disc3d` + ~6 `capsule3d`** | +| `drawScars` 1411 | `Object.entries(partDamage)` + `filter` per part every frame | 0-8 `capsule3d` | +| `drawOutfit` 1228 | shoes 2 × (3 `filledCapsule` + 2 `stroke`); sleeves 2 × 2 `filledCapsule`; shirt 2 quads; collar `filledDisc`; **daisy 16 `filledDisc`**; `drawSkirt` 1292: player = 6×4-node cloth, 3 passes × (stretch + `separate` vs thighs), 24 `projectPoint`, 15 quads + 23 `stroke`; kid = 2 tris + 3 strokes | **~10 `capsule3d` + 17 `disc3d` + ~80 `triangle3d`** | +| `drawHairline` 1100 | 19 points, 19-tri fan, 9 `filledCapsule`, 10 `stroke` | **9 `capsule3d` + ~40 `triangle3d`** | +| `drawBow(…,"loops")` | 8 tris + 2 `stroke` + 2 `filledDisc` | 12 `triangle3d` + 2 `disc3d` | +| `drawFace` 16407 → **`drawTrioFace` 16305** | spherical feature mapping; 5 `bezier()` each `Array.from(...).map(P)`; blush 2 ovals × 10 tris; per eye 2 ovals × 12 tris + 4 `filledDisc` + lid path 4 `filledCapsule` + 3 lash `filledCapsule` + brow 2; nose 2; lips 2 fans × 5 + 5 capsules; civilian kids = 4 `filledDisc` + 1 `stroke` | **~90 `triangle3d` + 8 `disc3d` + ~30 `capsule3d`** | +| `drawEmoFringe` 21119 (emo kids) | 7 tris + 1 capsule | — | +| `drawInventory` 16694, `drawHeldAxe` 21108 | `segments.find` ×3 by regex; axe = 1 `worldCapsule` + 2 × (`worldQuad` + `worldCapsule`) | 0-5 calls | +| `drawBubble` 17164 (blocking) | 33 `filledCapsule` + 1 `filledDisc` | — | + +Sum per LOD-0 skater ≈ 430 `triangle3d` + ~100 `capsule3d` + ~48 `disc3d` — consistent with the measured ~260 `capsule3d` / ~200 `disc3d` across 2 full figures + 3 cheaper civilian kids. + +### `drawSeatFirstPerson` 21401 (P1 inset, 4.2 ms) +- 2 `screenRect` (4 `triangle3d`), `P1 VIEW` `comicWrite`. +- `rebuildSeatInset` 21419 **every frame** walks `[pipeSurfaceMesh, pipeWallMesh, hallMesh, waterMesh?, ...cityBlocks.values()]`: per mesh 8 `local()` corner allocs for the bounds test, `new Float32Array(27)` camera, then **`sceneMesh`** — so the C side re-transforms and re-clips *every face of every park mesh a second time per frame* (that C time is inside the JS call and therefore inside `hostJsMs`). `SceneMesh` in `ScenePrimitives.inc` also allocates a `std::vector poly` per face and runs 5 `SceneClip` passes each allocating a fresh vector. +- Kids at 30 Hz (`kidsDue`): `runnerWorldGeometry` (cached) → per segment `face()` (front-fast path, else `clipPolygon` ×5 with `mixVertex` allocs) → `values.push` → `new Float32Array(values)` → `triangles3d`. + +### `gameSim` 13090-13519 (7.5 ms) — relevant to paint because it rebuilds poses paint then rebuilds again +- 13415 `updateParkKids` (8 kids, `activePlayers().some` per kid), 13416-13417 `updatePlayer` ×2, 13421 `updateHeavyWheelHits`, 13427 `updateMonowheel`, 13428 `updateParkSupply`, 13430 `updateWheelTurbo`, 13432 `updateSeatHeartbeat` (O(players × (players + balls + bullets)) `Math.hypot`), **13438 `resolveMelee` 11035 → `samples = players.map(p => sampleCombatBoxes(p, now))`** → `runnerWorldGeometry` **uncached** (`sharingRenderPoses` is false in sim, 14134) → full `buildRunnerWorldGeometry` per player per tick + `combatRect` object per segment; **13446 `updateBall` 10442** per ball → `boxesAt` → `sampleCombatBoxes` again (its own per-call `Map`, not shared with `resolveMelee`); 13451 `updateCameraDoll`. +- So `buildRunnerWorldGeometry` runs ≈ 2 (melee) + 2×balls (ball) + 2 (containFighters, paint) + 3 kids (paint) ≈ 9-12×/frame → the measured 2.9 ms/frame (~0.25-0.3 ms per build). + +--- + +## 2. Classification per phase + +| Phase | Output class | Cached today? | Recomputed every frame | +|---|---|---|---| +| pipe surface / front wall (18129/18153) | **static per map** | yes: `captureQuadMesh` → `nativeMeshBuffers` (Float32Array pair, built once) → `sceneMesh` | C re-projects + clips all faces per call; JS side: `mainNativeCamera` (1 alloc/frame), bounds nothing | +| hall (755), city blocks (21366), loops (679), water (20981) | **static per map** (keys rebuild only on pane/cube break, glass break) | mesh cached; **`mesh.capsules` are NOT** — replayed through JS `worldCapsule`→`worldSegment` (2 `toView` allocs + `projectView` + `clipSegmentBand`) → `capsule3d` every frame (~120 capsules) | yes for capsules; `drawDust` 28 particles fully recomputed | +| grid heat overlay (18322) | static-ish (decays) | no | iterates all cells; draws only hot | +| ledges, boosters, decals, pickups | static / slowly changing | no | `worldQuad` per item per frame, decals allocate spread objects | +| spot shadows (18600) | per-object | no | 3 `projectPoint` + 14 tris per caster | +| fighter pose (`buildRunnerWorldGeometry` 14413) | **per-object per sim tick** (pure function of player state + `frameNow` quantized to `replayTickUs`) | paint-only `renderPoses` Map 14130 keyed by player, cleared each paint; sim never shares | rebuilt 9-12×/frame across sim + paint | +| `constrainLimbs` + `projectRunnerWorldGeometry` | per-object per frame (camera-dependent) | no | full copy + ~29 projections + allocs per figure | +| limbs/face/outfit/hair/skirt/bow (13799, 16305, 1228, 1292, 934, 1145, 1100) | per-object per frame, **screen-space**, but shape is a function of (pose, facing, yaw bucket, blink, cameraScale) | no | full re-tessellation in JS every frame; cloth/Verlet/Bézier all in JS | +| skateboard (18712) | **rigid body per object** — deck/trucks are a fixed mesh in board-local space; only the frame (`skateFrame` 15531) changes | no | 30 `worldQuad` + 20 `worldCapsule` + 16 `disc3d` rebuilt per board per frame | +| P1 inset (21401) | meshes static per map; kids per-object | kids buffer at 30 Hz (`seatInsetBuffer`); meshes **re-submitted every frame** | yes for meshes (C side re-clips everything) | +| HUD text (`seatHudText`, `drawSeatAction`, fps/score) | **screen-space** | `drawFreeskateSpeed` had a 100 ms string cache (5288-5299) but is short-circuited to `drawSeatPlayerHud` at 5265 which caches nothing | strings rebuilt + `handleWidth` (`[...text].reduce`) per frame; native side DirectWrite `DrawText`/`DrawGlyphRun` per string (App.cpp 2610-2660) | +| `acFeed = ac()` 20281, `fighterProfile()` per player, `displayTheme()`/`losAngelesSun()`, 20 `mixColor` | constants / once-per-second data | no | every frame | + +--- + +## 3. JS→native boundary and marshalling + +Bindings (QuickJsEngine.cpp:960-1006): `triangle3d`(12 args), `triangles3d`(Float32Array, count ≤ 8192), `disc3d`(7), `capsule3d`(9), `sceneMesh`(3 Float32Arrays), `sprites3d`, `texturedTriangles3d`, `themeSprite`/`themeQuad`, `box`/`line` (D2D, composited *under* the triangle pass — `DrawVectorBackground` App.cpp:2403), `write` (CPU block glyphs), `systemWrite`/`ywftWrite`/`comicWrite` (DirectWrite, `kMaxSystemDraws = 128`), `runtime`, `capabilities`, `ac`. + +Per-call cost on the C side is small: `Triangle3d` (209-228) does 9 `JS_ToFloat64` + 3 `JS_ToInt32` (values are already doubles/ints — tag checks), one `std::isfinite` pass, and a `push_back` into `m_frameTriangles`. `SceneCapsule` tessellates 2 + 2n tris (n = 3…12) in C; `SceneDisc` n = 8…28. `SceneMesh` does the full transform + 5 Sutherland-Hodgman clips per face in C, with a heap `std::vector` per face and per clip pass. + +**The `emitTriangle` comment (oskiewar.js:2145-2153) checks out.** It says ~2100 faces/frame, a Float32Array staging buffer cost 5.7 ms vs 1.8 ms for direct `triangle3d`, and the batched host call itself was 0.05 ms. That is consistent with how quickjs-ng executes: a 12-argument native call is one `OP_call` with the arguments already on the operand stack, then 12 cheap tag conversions in C; whereas `buf[i+k] = v` in JS is, per element, index arithmetic bytecode + `OP_put_array_el` → `JS_SetPropertyValue` typed-array path → float conversion + store, i.e. ~12 interpreted property stores per face plus the loop counter. Twelve interpreted stores beat one interpreted call, and the C-side reading of the buffer is free. Corroborating evidence inside the same file: `rebuildSeatInset` avoids typed-array element writes and instead `values.push(...)` into a plain Array and does one `new Float32Array(values)` (21478) — a bulk copy in C — which is the right way to batch *if* batching were the goal. + +**Weighing it:** the comment's conclusion (don't batch in JS) is correct, but its framing invites the wrong fix. Neither direct calls nor batching removes the real expense, which is *producing* the ~2100 faces' coordinates in interpreted JS (projection, Bézier, Verlet, IK, string-keyed lookups, allocation). The 1.8 ms is the floor for emitting 2100 faces from JS regardless of API shape. The only architectural moves that cut below it are ones where JS emits O(objects) calls and C++ produces the faces: retained meshes by handle, primitives that tessellate natively (`capsule3d`/`disc3d` already do this and are the right shape), and a native affine/model transform so screen-space or object-space geometry can be uploaded once and re-placed per frame with a handful of floats. + +Hidden boundary costs found: +- `sceneMesh` for the inset: every mesh every frame, with C-side per-face heap allocation (ScenePrimitives.inc `SceneMesh`/`SceneClip`). This C time is billed to `hostJsMs`. +- `ac()` every `gamePaint` (20281) → `AcData` QuickJsEngine.cpp:847 builds a nested object graph per frame. +- `runtime()` is only snapshot-cached in paint (21240-21243); every `runtime()` in sim (`updateParkKids`, `updateSeatHeartbeat`, `fighterAnimationPhase` 13982, etc.) rebuilds a ~20-property object natively. +- `themeReady()` per frame in `refreshPhotoTheme` 13711 (cheap). + +--- + +## 4. Top 10 JS-time sinks and the cheapest structural move for each + +1. **`buildRunnerWorldGeometry` 14413 — 2.9 ms/frame, built 9-12× (sim + paint).** + Sinks inside: `partForRole` 14478 (8 string tests per segment), `hasPart` 10069 (`includes` per segment), `segments.find` ×2 per leg 14807-14808, `constrainLimbs` 14880 (`{...segment}` copy + `find` with regex per bone), `fighterAnimationPhase` 13981. + Move: make the pose a **per-player per-tick memo** keyed on `animation.frameNow` (already quantized to `replayTickUs`) + the handful of state fields the pose reads; populate `renderPoses` (or a sim-side twin) from `resolveMelee`/`updateBall` so paint's `containFighters`/`drawRunner`/`rebuildSeatInset` reuse the sim's build. Share one `sampleCombatBoxes` result between `resolveMelee` 11038 and `updateBall.boxesAt` 10464. Replace `partForRole` with a static `Map` built once per (facing, itemHand). Expected: builds/frame 9-12 → ≤5, i.e. ~1.5-2 ms. + +2. **`drawCurvedLimbs` 13799 (+ `drawSkeletonSegments`/`drawFighterSilhouette`) — 2.1 + 2.2 ms.** + Per limb per frame: Bézier eval 4-8 steps, `stroke` per step × 2 passes (2 `triangle3d` each), `filledDisc` caps, plus `find` + regex to pair bones. + Move: **native `curve3d(x0,y0,c1x,c1y,c2x,c2y,x1,y1,z,width,r,g,b)`** (a cubic ribbon tessellated in C, same family as `capsule3d`) — one call per limb per pass instead of ~9 calls + point allocations; 4 limbs × 2 passes = 8 calls per figure. Pair bones once per pose (store `lowerIndex` on the upper segment in `buildRunnerWorldGeometry`). + +3. **`drawOutfit` 1228 + `drawSkirt` 1292 — 2.6 + 1.4 ms.** + Skirt cloth: 24-node Verlet, 3 passes × (stretch + thigh `separate`), 24 `projectPoint`, 15 quads + 23 strokes. Daisy: 16 `disc3d`. Shoes: 6 `capsule3d` + 4 strokes. + Move: step the skirt cloth at 30 Hz (the P1 inset already sets the precedent at 21410) and drop pleat resolution when `head.radius * cameraScale()` is small; emit the skirt as one `triangles3d` buffer filled via `values.push` + `new Float32Array` (the seat-inset pattern, not element writes). Daisy/shoes/collar are rigid in torso-local space → one `sprites3d` point sprite (binding exists, 292) or a tiny retained screen-space mesh with a native affine (see 6). + +4. **`drawTrioFace` 16305 + `drawFace` 16407 — 0.75 + 0.85 ms.** + 5 `bezier()` per face each `Array.from().map()`, spherical `P()` per feature point, ~90 tris + 30 capsules + 8 discs. + Move: the face is a pure function of (facing, spin-yaw bucket, blink, roll, radius). Memoize a unit-radius triangle list per (yaw/32, blink) in a plain Array → `new Float32Array` once, then draw with a **native affine**: add `triangles3dAt(buffer, count, tx, ty, scale, depth)` (C applies `x*scale+tx`) so the per-frame JS is one call. Kids already take the 5-call civilian path. + +5. **`drawSkateboard` 18712 + `drawBoardRunningGear` 18682 — 1.9 ms per frame.** + Deck = 30 `worldQuad` (each 8 projections + `litQuadColor` alloc), trucks = 20 `worldCapsule`, wheels = 16 `disc3d` + 8 tris; all rigid in board-local space, only `skateFrame` 15531 changes. + Move: capture the deck + trucks once with `captureQuadMesh` in board-local coordinates and **add a model matrix to `sceneMesh`** (extend the 27-float camera buffer by 12 floats: 3×3 rotation + translation, or a `sceneMeshAt(verts, faces, camera, model)` binding). One call per board. Wheels: 4 `disc3d` stay (they roll) or become 4 `sprites3d`. + +6. **Static park geometry re-submitted every frame — `drawQuadMesh` 21184 (3.4 ms), `drawIndoorHall` 755 (2.1), `drawUnderPipe` 20981 (1.5), plus the inset's second pass.** + JS cost is not `sceneMesh` itself; it is (a) `mesh.capsules` replayed through `worldCapsule`→`worldSegment` (2 object allocs + clip per capsule, ~120/frame: hall cubes 21181-21186 rows/diagonals/edges, coping, 60 glass-pane highlights 21015, chain links), (b) `drawDust` 731 (28 `projectPoint` + 56 tris), (c) per-block bounds and camera buffers, and (d) C-side re-clipping of every face of every mesh twice per frame (main + inset). + Move: **retained mesh handles** — `sceneMeshUpload(id, verts, faces)` once, `sceneMeshDraw(id, camera)` per frame (27 floats). Fold capsules into the mesh at capture time as world-space segments in a third stream that C tessellates *after* projection (`captureQuadMesh` already intercepts `worldCapsule` at 21175; just stop replaying them in JS). Merge pipeSurface + pipeWall + hall + water + visible city blocks into one draw. `drawDust` → `sprites3d` (exact fit: 28 point sprites) or 30 Hz. In C, replace the per-face `std::vector` churn in `SceneMesh` with fixed stack arrays (poly ≤ 4+5 verts). + +7. **`drawSeatFirstPerson`/`rebuildSeatInset` 21401/21419 — 4.2 ms.** + Every frame: per mesh 8 `local()` allocs + `new Float32Array(27)` + `sceneMesh` (C re-transforms and re-clips every park face for a 320-px inset); kids 30 Hz `face()`/`clipPolygon`. + Move: with 6 done, the inset is N `sceneMeshDraw(id, insetCamera)` calls; C can additionally cache the inset's clipped output at 30 Hz. Cheapest immediate: submit the inset's meshes on alternate frames only (the kids already are) — halves the 4.2 ms without touching native — and skip meshes whose 8 bounds corners all fail the frustum test *before* allocating the camera buffer. + +8. **HUD text — `drawHudStatusTray` 19744 (1.6 ms) + `drawSeatPlayerHud` 21137 + `drawSeatAction` 21148.** + 12-16 `comicWrite`/frame; each string rebuilt (`Math.round(...) + ' mph'`, `'P1 ' + score + ' : ' + ...`), `handleWidth` = `[...text].reduce(comicGlyphAdvance)` per call; native side lays out with DirectWrite per string per frame (App.cpp 2610-2660; the file's own note at 5281: "the console rasterizes every new string … seven milliseconds"). `drawFreeskateSpeed` 5264 had a 100 ms string cache but is bypassed at 5265. + Move: cache the seat HUD strings at 10 Hz per player (copy the 5288 pattern), memoize `handleWidth(text, size)`, draw shadows only on the large reading. Later: a native `hudText(id, ...)` that keeps the DirectWrite layout until the string changes. + +9. **`gamePaint` prologue 20248-20345 + renderables 20520-20549 — allocation churn.** + `acFeed = ac()` every frame (native object graph); `fighterProfile(player.name)` per player; `displayTheme()`→`losAngelesSun()`; ~20 `mixColor` arrays; 5 array spreads + `.sort` closure; 3 `projectPoint` per renderable (each 2 allocs); `worldSegment` allocs 2 per capsule; `projectPoint` never takes an `out`. + Move: call `ac()` at 1 Hz; compute the palette once per `visualTheme` change; give `projectPoint` a scratch `out` (as `worldTriangle`/`worldQuad` already do at 2305-2306) and use it in `projectRunnerWorldGeometry`, `drawSpotShadow`, decals, dust; keep a persistent `renderables` array and sort by a precomputed `depth` field. + +10. **`gameSim` pose and box rebuilds — 7.5 ms, of which `updateBall` 2.4 + `resolveMelee` 1.2.** + `resolveMelee` 11038 samples *all* players' boxes every tick with an uncached pose; `updateBall` 10464 samples again per ball; `sampleCombatBoxes` builds a `combatRect` object per segment; `updateSeatHeartbeat` 21517 is O(P × (P + balls + bullets)) with `Math.hypot`; `runtime()` in sim is uncached (21240 caches paint only). + Move: one `combatBoxes` cache per tick keyed by (pad, now) shared by melee, balls and `drawDebugHitboxes`; extend the `runtime()` snapshot wrapper to `sim`; skip `resolveMelee` for a player with no active attack (`attacking.hit.length` is checked only after the sample is built). + +### Order of operations that gets to <16 ms with the least native work +1. Pose memo shared sim→paint (item 1, 10): pure JS, ~2-3 ms. +2. Stop replaying `mesh.capsules` in JS and submit the inset's meshes on alternate frames (items 6, 7): pure JS, ~3-4 ms. +3. Skirt/ribbon/ponytail sims at 30 Hz, HUD strings at 10 Hz (items 3, 8): pure JS, ~2 ms. +4. Native: `sceneMeshUpload/Draw` with a model matrix (items 5, 6, 7) and `curve3d` (item 2) — this is what removes the O(faces) JS floor the `emitTriangle` comment measured, and turns each figure into ~30 native calls instead of ~580. diff --git a/xbox/research/research-world.md b/xbox/research/research-world.md new file mode 100644 index 0000000000..aa9e397140 --- /dev/null +++ b/xbox/research/research-world.md @@ -0,0 +1,240 @@ +# oskiewar world rendering — retained native scene research + +Sources: `/Users/jas/ac-worktrees/oskiewar-day-perf/xbox/live/oskiewar.js` (JS, 21546 lines), `/Users/jas/ac-worktrees/oskiewar-perf/xbox/native-bios/{ScenePrimitives.inc,QuickJsEngine.cpp,App.cpp}`, `/Users/jas/ac-worktrees/oskiewar-perf/xbox/runtime/include/ac/runtime.hpp`. Line numbers below are from those files. Read only; nothing edited. + +## Findings that change the plan + +1. **`drawQuadMesh`'s 3.4 ms on console is mostly NOT the meshes — it is capsule replay in JS.** The native branch (oskiewar.js 21186-21189) hands the faces to `sceneMesh` once, but first loops `mesh.capsules` and calls `worldCapsule` for each (21187). `hallMesh` carries 56 capsules (6 funboxes × 9 at 799-806 + 2 coping at 811-814); `waterMesh` carries 62 (60 bottom-glass panes at 21010 + 2 pond waterlines at 21001). Each `worldCapsule` (17875) → `worldSegment` (17851: 2 `toView` + 2 `projectView`, all allocating fresh `{x,y,z}` objects, plus `clipSegmentBand`) → `filledCapsule` → `capsule3d` (9-arg native call). ~120 of those a frame. This is the biggest single win and it needs only a capsule list inside the retained handle. + +2. **The P1 inset's bounds cull never fires on Xbox.** `rebuildSeatInset` (21446) tests `if(mesh.bounds)`, but `mesh.bounds` is only computed lazily in the JS fallback branch of `drawQuadMesh` (21191), which the native branch returns before (21189). On console `mesh.bounds` is `undefined`, so every frame the inset sends `pipeSurfaceMesh`, `pipeWallMesh`, `hallMesh`, (`waterMesh`), and **all ten** `cityBlocks` through `sceneMesh` a second time, uncullled, each with a freshly allocated `new Float32Array(27)` camera (21459). The main view culls city blocks by x-span (21368) but the inset does not. + +3. **`sceneMesh` (ScenePrimitives.inc 51-73) re-validates the whole index stream on every call (line 64) and heap-allocates per face**: `std::vector poly{...}` (65) plus five `SceneClip` calls each returning a new vector (66, 68). ~700 faces/frame × 6 allocations. A retained handle validates once at upload and clips into a fixed `ScenePoint[9]` (a quad clipped by 5 planes has at most 9 vertices). + +4. **The measured numbers nest.** `drawHudStatusTray` (19744) calls `drawParkSupply`, `drawUnderPipe` and `drawWaterSplashes` first (19746-19748); its 1.6 ms is ~1.5 ms `drawUnderPipe` (= `waterMesh` = 62 capsules) and ~0.1 ms of actual tray. `drawIndoorHall` (755) includes `drawCityStreet` (5 `drawQuadMesh`) + `hallMesh` + `drawDust`. So "drawQuadMesh 3.4 across 13 meshes" already contains most of the 2.1 + 1.5. + +5. **`globalLight` is a constant** (1945: `normalize3({x:-.42,y:1,z:-.28})`), so face lighting (`.72 + max(0, -n·L) * .28`, ScenePrimitives.inc 69, litQuadColor 17905) can be baked at upload. The camera's 27 floats then only need 24; keep the slots for compatibility. + +6. **The photo texture path is dead for the world.** `photoSurface` (13781) returns `false` for `"wall"`, `"concrete"` and `"skirt"` (13784) before touching `themeQuad`, so no terrain face is ever textured. Retained meshes need no UVs in phase 1. Textured world faces would come back through `texturedTriangles3d` (QuickJsEngine.cpp 413-465; 18 floats/tri; cap `kMaxTexturedTriangles` 2048, App.cpp 2754). + +7. **One 8192-triangle budget for everything** (`kMaxTriangles`, App.cpp 2753; `on_triangle` drops silently past it, 371-373). `sceneMesh` output, fighters, HUD boxes and the inset all share it. The inset caps itself at 6000 tris (21429). `AC_NATIVE_FRAME ... trianglesDropped=` is logged once at first frame only (`m_loggedTextFrame`, 2678-2698), so a mid-round overflow is invisible. + +8. **When a roof pane breaks, the whole hall falls back to the per-frame JS path** (757: `if(roofShards.length){drawIndoorHallGeometry(ground);return;}`) for as long as shards exist — 91 quads + 56 capsules re-projected in JS per frame. With a retained hall handle the shards (a separate loop at 826-836) draw on their own. + +9. **Freeskate HUD bypasses its own 10 Hz string cache.** `drawFreeskateSpeed` (5264) returns on line 5265 straight into `drawSeatPlayerHud` (21130), which formats `measure`/`verb`/`item` strings every frame (21136-21140); the 100 ms cache it skips is at 5284-5296. Each new string is a `JS_ToCString` + `std::string` + `std::wstring` + a fresh DirectWrite glyph run (`QueueSystemWrite` 467-486; `drawPackagedFont` App.cpp 2618-2655 calls `GetGlyphIndices` + `GetDesignGlyphMetrics` per text per frame). + +## 1. Inventory + +Legend: **S** = static per map (built once, keyed), **D** = dynamic world geometry, **SS** = screen-space. "JS/frame" = what JS does each frame on console today. Native endpoint = where the pixels go. + +### Backdrop and room + +| Element | Class | Where | Built | JS/frame | Native call | +|---|---|---|---|---|---| +| clear + sky bands/streaks | SS | `gamePaint` 20346 `wipe`; `drawSkyAtmosphere` 18418 (survival only) | per frame | 6 `box` + 11 `line` (survival only; park: nothing) | `wipe`, `box`, `line` → CPU frame / D2D under the 3D pass | +| room walls / containment lines | D | `drawRoomSurfaces` 18215 (station + survival only; indoor returns nothing) | per frame | survival: 1 wall + 4 side quads + 2 `worldLine`; station: `drawGridOverlay` + 4 `worldLine` | `triangle3d` per tri; `line` | +| grid heat overlay (station) | D | `drawGridOverlay` 18322 | per frame | up to 16 quads per hot tile, 57 seam quads (climb only) | `triangle3d` | + +### Terrain (three passes) + +| Element | Class | Where | Built | JS/frame | Native | +|---|---|---|---|---|---| +| back wall | S (outdoor) | `drawTerrainBackWall` ~18186 → `terrainPass` 18075 | profile thinned per LOD at map change (`rebuildTerrainDrawProfile` 17983) | project each profile point once (`terrainVertex` 18040), 1 quad/segment in span; indoor: returns | `triangle3d` ×2 per quad | +| surface | S | `drawTerrainSurface` 18129: indoor → captured `pipeSurfaceMesh` (~36 quads: 2 transitions × 16 + 4 flats); outdoor → `terrainPass` | key `gridCols:near:far:color` | indoor: `drawQuadMesh`; outdoor: per-point projection over `terrainSpan` (8785, ±2600 apron) | `sceneMesh` / `triangle3d` | +| front wall / skirt | S | `drawTerrainFrontWall` 18153: indoor → `pipeWallMesh` (1 big + ~36 quads); outdoor → `terrainPass` with 1 row on console (18180) | keyed | as above | `sceneMesh` / `triangle3d` | +| grass | D (deterministic) | `drawTerrainGrass` 18385; skipped in space (`!space`, 20390) | per frame | 100 tufts × 2 `worldLine` → each 2 `toView` + 2 `projectView` | `line` (D2D, under everything) | + +Terrain camera-space math in JS: `FightCamDoll.toView` 2071, `projectView` 2084, `prepare` 2049. `bandContains` 2298 uses `guardBand = 1` (1936) i.e. ±1 viewport; `cameraNear = 8` (1931). + +### Park furniture (indoor hall) + +| Element | Class | Where | Built | JS/frame | Native | +|---|---|---|---|---|---| +| funboxes (6 cubes) | S (rebuild on break) | `drawIndoorHallGeometry` 762-806 | in `hallMesh`, key = ground + pane/cube broken bits (758) | 30 quads via `sceneMesh`; **54 capsules replayed in JS** | `sceneMesh` + 54× `capsule3d` | +| pipe coping (2) | S | 811-814 | in `hallMesh` | 2 `worldCapsule` | `capsule3d` | +| glass roof: 20 panes × (pane + glint + frame) + mid rail | S (rebuild on break) | 816-830 | in `hallMesh` (~61 quads) | `sceneMesh` | `sceneMesh` | +| roof shards | D transient | 831-838 | per frame | `projectPoint` + 1 `screenTriangle` each; **forces whole hall to JS path while any exist (757)** | `triangle3d` | +| dust (28 specks) | D cosmetic | `drawDust` 731 | per frame | 28 `projectPoint` + 56 `screenTriangle` | `triangle3d` | +| city street: 10 blocks × 29 quads | S | `drawCityStreet` 21366 / `buildCityBlock` 21374 | lazily captured per block into `cityBlocks` Map | x-span cull (21368) then ~5 `drawQuadMesh` | `sceneMesh` ×5 | +| loop track (indoor: 1 loop r=260; park: 3) | S per LOD | `drawLoopTrack` 679 → `loop.mesh`, key includes `sceneLod()` (681) | 64 seg × 4 quads = 256 quads at LOD 1 (32/16 seg + 1 side at LOD ≥2) | `drawQuadMesh` | `sceneMesh` | +| under-pipe water/tunnel: 30 column quads + pond sheet + bottom + 60 glass panes × 2 quads | S (rebuild on glass break) | `drawUnderPipe` 20981 / `drawUnderPipeGeometry` 20987; key = `bottomGlass` bits + gridCols | `waterMesh` (~153 quads) | `drawQuadMesh`; **62 capsules replayed in JS** | `sceneMesh` + 62× `capsule3d` | + +### Park furniture (outdoor course, not captured) + +| Element | Class | Where | JS/frame | Native | +|---|---|---|---|---| +| ponds: sheet + 16 front + 16 waterline quads | S-shaped, drawn dynamic | `drawSkateParkFeatures` 1457-1478 | 33 `worldQuad` per visible pond | `triangle3d` | +| boost pads: 6 chevrons × 2 capsules | S | 1480-1486 | up to 12 `worldCapsule`/pad | `capsule3d` | +| chains: 1 bar + 16/linkStep links | D (Verlet) | 1488-1499 | `worldCapsule` per link | `capsule3d` | +| booster pads (station) | D (pulsing tint) | `drawBoosterPad` 18405 | 1 quad per pad | `triangle3d` | +| ledges/decks | S | `gamePaint` 20407-20422 | 1 `worldQuad` + 1 `worldLine` each | `triangle3d`, `line` | +| survival lava | D | `drawSurvivalLava` 18279 | 2 quads + 9 `worldLine` | | + +### Decals and marks + +| Element | Class | Where | JS/frame | Native | +|---|---|---|---|---| +| decals (≤45 drawn, arc/skid/blood/ding/wheelmark/chip) | D append-only, ages | `drawDecals` 833 | arcs: 4-8 `worldQuad`; skids: 2 `worldQuad`; fans: 3 `projectPoint` + N `screenTriangle` from a cached shape | `triangle3d` | +| blood drops | D transient | 893-902 | 2 `projectPoint` + 1 tri each | `triangle3d` | +| spot shadows (fighters, balls) | D | `drawSpotShadow` 18600 | 3 `projectPoint` + 14 `screenTriangle` | `triangle3d` | + +### Props + +| Element | Class | Where | JS/frame | Native | +|---|---|---|---|---| +| skateboard (deck) per rider/loose board | rigid body, D pose | `drawSkateboard` 18712: 7 stations × 4 `worldQuad` + 2 caps + crack capsules | ~30 `worldQuad` (each `litQuadColor` + 4 `toView` + 4 `projectView`) | `triangle3d` ×60 | +| trucks (2) | rigid | `drawTruck` 18654: 12 capsules at full scale, 3 at `scale<.65` | 24 `worldCapsule` | `capsule3d` ×24 | +| wheels (4) | rigid + spin mark | `drawBoardRunningGear` 18682 | 4 × (`projectPoint` + 4 `filledDisc` + 2 `screenTriangle`) | `disc3d` ×16, `triangle3d` ×8 | +| monowheel | rigid | `drawMonowheel` 20889: 12 seg × (1 quad + 4 `worldTriangle`) + 8 quads | ~68 faces | `triangle3d` | +| balls (soccer/beach) | D | `drawBall` ~18760 | screen-space fans | `disc3d`, `triangle3d` | +| pickups: axe, SMG, saber, grenade, body trees | D bob | `drawParkSupply` 20936, `drawGunPickup` 18896, `drawSaberPickup` 18965, `drawGrenadePickup` 19080, `drawBodyTree` 9669 | few quads/capsules or `themeSprite` under photo theme | mixed | +| drone / drop | D rare | `drawParkSupply` 20938-20944 | 1 capsule + 2 quads / 1 quad + 2 capsules | | +| turbo particles, splashes, impacts | D transient | `drawTurboParticles` 21345, `drawWaterSplashes` 21047, `drawImpacts` 20112 | `projectPoint` + `filledDisc`/`filledRing` | `disc3d`, `triangle3d` | + +### Crowd + +| Element | Class | Where | JS/frame | Native | +|---|---|---|---|---| +| parkKids (8, indoor freeskate only) | D characters | `resetParkKids` 21262 (cloned from `players[0]` via JSON), `updateParkKids` 21286 (sim); drawn in `gamePaint` 20544-20547 as `renderables` → `drawRunner` 17058 when on-screen | full runner pipeline per visible kid; `figureLod` 16978 tiers them by screen height | fighter path (character side) | +| kids in P1 inset | D | `rebuildSeatInset` 21478-21500, 30 Hz | `runnerWorldGeometry` per kid **again** (not shared with the main view's solve), ~20 segments × `face()` (4 object literals + clip checks + 12 `values.push`), head/hair/eyes/bow discs (10 tris each), then `new Float32Array(values)` | `triangles3d(seatInsetBuffer, count)` every frame (21413) | + +### P1 first-person inset (4.2 ms) + +`drawSeatFirstPerson` 21401: 2 `screenRect`, `rebuildSeatInset(rect, kidsDue)` (21411), `triangles3d` replay, 1 `screenRect`, 1 `typeWrite`. Per frame in `rebuildSeatInset` on console: for each of `[pipeSurfaceMesh, pipeWallMesh, hallMesh, waterMesh?, ...cityBlocks.values()]` (21444): bounds test (dead, finding 2), `new Float32Array(27)` camera (21459), `sceneMesh`. Inset camera layout (21459): `[eye.x,eye.y,eye.z, -fz,0,fx, fx*.12,-1,fz*.12, fx,.12,fz, rect.cx,rect.cy, 0, focal=rect.w*.68, 1, near=12, rect.x,rect.y,rect.x+w,rect.y+h, -1.497, .000002, L.x,L.y,L.z]`. + +### HUD (screen space) + +| Element | Where | JS/frame | Native | +|---|---|---|---| +| seat HUD (freeskate): measure/verb/item per player + heart + FPS + P2 score | `drawSeatPlayerHud` 21130, `drawSeatHeart` 21528, `drawHudStatusTray` 19770-19775 | ~8 `typeWrite` (each = shadow + ink = 2 `comicWrite`), strings rebuilt every frame | `comicWrite` → D2D `DrawGlyphRun` (render side) | +| seat action word | `drawSeatAction` 21150 | 2 `typeWrite` at `hudDepth` | | +| nameplates (fight modes) | `drawPlayerHandle` 17417 | **per glyph** `typeWrite`, 2 passes (shadow + colour) | one `SystemText` per glyph; cap `kMaxSystemDraws` 128 (App.cpp 2752) | +| command stream, inventory, stats | 17604, 17549, 17522 | 2 `typeWrite` per entry | | +| status tray (MIDI piano) | `drawStatusPiano` 19735 | 6 `hudBox` | `triangle3d` at `hudDepth -1.48` | +| frame meter (debug) | `drawFrameMeter` 17321 | up to 2×120 `hudBox` | `triangle3d` | +| control legend / stick gates | `drawControlLegend` 16110 (returns in freeskate) | `filledDisc` ×3 + `typeWrite` per live stick | `disc3d` | +| net health, QR | `drawNetHealth` 20043, `drawSpectatorQr` 20067 | `screenRect` runs | `triangle3d` at -1.43 | +| touch controls | `drawTouchControls` 16219 (touch only) | capsules + discs | | + +HUD depth lanes: world faces `-1.4 + vz*.000175` (fallback 21203, C++ `m[22]+p.z*m[23]`), debug `-1.4`, screen UI `-1.42`, net/QR `-1.43`, stats `-1.445`, `hudDepth = -1.48` (2184), inset `-1.497 + z/6000*.012` window, inset frame `-1.482`/`-1.499`. Shell maps `(z+1.5)/3` into the depth buffer, `LESS_EQUAL` (App.cpp 797, 831). `box`/`line` go to the CPU frame/D2D under every triangle (2440-2582 / `DrawVectorBackground` 2404), which is why `hudBox`/`hudLine` (2185/2192) route through `screenRect` on console. + +### Camera + +`FightCamDoll` 2023; `cameraDoll.prepare()` at 20364 once per paint (also on `dirty`). `mainNativeCamera` 21508 rebuilds a `new Float32Array(27)` whenever `monotonicUs` changes, i.e. once per frame: `[pos xyz(0-2), right(3-5), up(6-8), forward(9-11), centerX(12), centerY(13), orthoScale(14), focal(15), perspective(16), cameraNear(17), bandMinX(18)=0, bandMinY(19)=0, bandMaxX(20)=viewWidth, bandMaxY(21)=viewHeight, depthBase(22)=-1.4, depthSlope(23)=.000175, light xyz(24-26)]`. C++ projection (ScenePrimitives.inc 67): `k = m[14] + (m[15]/z - m[14]) * m[16]`; `x = m[12] + vx*k`, `y = m[13] - vy*k` — identical to `projectView`'s lerp of ortho and perspective. Note the C++ band is the exact viewport, the JS band is ±1 viewport (`guardBand`); harmless, C++ clips. + +Native compositing order (App.cpp `Render` 2427-2700): CPU frame (rects, images, lines, block text) or D2D fast path → upload as scene texture → clear depth (2584) → `DrawGpuTriangles` 2587 (all `Triangle`s, one vertex map, 819-857) → `DrawGpuTexturedTriangles` → `DrawGpuSprites` → `DrawGpuThemeQuads` 964-1013 (6 passes by asset, soft passes depth-test without write) → D2D system texts/glyphs on top (2591-2675) → `DrawPostProcess` 2676. JS is timed as `hostJsMs` (sim + paint, 560-594); D2D text and the vertex map are `renderCpuMs`. + +## 2. Retained native scene + +### API (phase 1 shape, all gated on `typeof meshUpload === "function"`) + +``` +meshUpload(vertices: Float32Array, faces: Float32Array, capsules?: Float32Array) → handle:int // once per map/key +meshDraw(handle, camera: Float32Array(27), tintR?, tintG?, tintB?) → trianglesEmitted // per view +meshFree(handle) +``` + +Layouts (keep today's so `nativeMeshBuffers` 21504 needs no change): +- vertices: `[x,y,z]` × N (world units). +- faces: `[i0,i1,i2,i3, r,g,b, nx,ny,nz]` × F (10 floats, quads only — `captureQuadMesh` 21172 only intercepts `worldQuad`; `worldTriangle` calls inside captured geometry are not captured today either). +- capsules (new): `[x1,y1,z1, x2,y2,z2, worldWidth, r,g,b, depthBias]` × C (11 floats). Today `captureQuadMesh` stores `args[6] /= cameraScale()` (21175) and `drawQuadMesh` multiplies back by the current `cameraScale()` (21187), so `worldWidth` is exactly that stored value; C++ computes `widthPx = max(1, worldWidth * m[14])` (orthoScale = `cameraScale()`), projects both ends through the same camera with the `worldSegment` near-clip (17851-17857), sets depth `min(za,zb)*m[23] + m[22] + depthBias` (bias −.004, 17876), and reuses `SceneCapsule`'s emitter (ScenePrimitives.inc 37-50) with screen endpoints. +- camera: the 27 floats above, unchanged, so `mainNativeCamera` and the inset camera work as-is. Make both reuse one preallocated `Float32Array(27)` each (27 element writes/frame is fine; `new Float32Array` per mesh per frame is not). + +C++ side (`ScenePrimitives.inc`): a `struct RetainedMesh { std::vector verts, faces, capsules; float aabb[6]; std::vector litColor; }` table in `CallScope`/engine; upload copies (input ownership stays with QuickJS, as `SceneFloats` already notes), validates indices once (move line 64's loop here), computes the AABB, and pre-lights each face with the constant light (finding 5) into `litColor`. `meshDraw`: transform the 8 AABB corners, reject if all behind near or all outside one band edge (this is exactly the JS test at 21194-21199, moved), then the existing per-face loop with `ScenePoint poly[9]` on the stack instead of vectors (65-68), and `points` as a per-engine scratch `std::vector` that is `resize`d, not reallocated. Tint multiplies `litColor` (booster pad pulse, theme light/dark mixes like `hall.wall` fog — optional). + +Phase 2 adds: +``` +sceneDraw(camera, viewMask) → emitted // draws every resident handle whose flags & viewMask, C++ AABB cull +meshFlags(handle, flags) // bit0 main view, bit1 inset, bit2 hidden +meshDrawAt(handle, camera, x,y,z, yaw,pitch,roll, scale, tintR,tintG,tintB) // instanced rigid prop +``` + +### What JS still computes per frame + +- The 27-float camera (once; already cached on `monotonicUs`) and the inset camera (eye from `players[0]` pose, 21420-21422). +- Which handles are resident/visible: key changes only (`pipeSurfaceKey`, `pipeWallKey`, `hallMeshKey`, `waterMeshKey`, `loop.meshKey`, `cityBlocks` membership). On a key change: recapture in JS (unchanged), `meshFree(old)`, `meshUpload(new)`. These are rare (glass/roof/cube breaks, LOD tier change for loops, map change). +- Dynamic world geometry that stays on the JS path: decals (≤45 quads/frame; candidate for an append-only handle later), roof shards, dust, particles/splashes/impacts, spot shadows, chains, grass lines, ledges (4 quads), booster pads, drone/drop, pickups. None of these is large. +- Props drawn as instances (phase 2): per board/wheel a `meshDrawAt` with pose from `skateFrame`/`spunSkateFrame` (18716) — JS computes the 6-DOF pose, not the 30 quads. +- Text and HUD (section 4). + +### Culls that move into C++ + +| JS cull today | Where | C++ replacement | +|---|---|---| +| 8-corner bounds test (fallback only) | `drawQuadMesh` 21191-21199, `rebuildSeatInset` 21446-21450 | AABB vs near plane + band in `meshDraw`/`sceneDraw` | +| `bandContains` × 4 fast path / `clipScreenBand` | `drawQuadMesh` 21207-21209, `worldQuad` 17928-17931 | already `SceneClip` against `m[18..21]` (ScenePrimitives.inc 68); add the all-inside fast path before clipping | +| per-face all-left/right/top/bottom rejection | 21204 | same as above, before allocating anything | +| `drawCityStreet` x-span (`cameraCenter ± cameraWidth*1.4`) | 21368 | AABB cull in C++; JS can still skip `meshUpload` for blocks never seen | +| near-plane fast path (`vz >= cameraNear` × 4) | 17926, 21207 | already in `SceneClip`; make it a compare before the clip walk | +| `terrainPass` span filter (`inSpan`) | 18082-18085 | chunked terrain handles (one per 8 tiles) + AABB | +| skateboard `nearSide` face ordering | 18732 | depth buffer does it; draw all faces | + +### Two views + +One resident set drawn twice: `sceneDraw(mainCamera, 1)` in `gamePaint` where the park draws today (20384-20387), and `sceneDraw(insetCamera, 2)` inside `drawSeatFirstPerson` replacing the mesh loop at 21444-21461. `waterMesh` gets flags `1 | (player.underPipe ? 2 : 0)` (today's `...(player.underPipe?[waterMesh]:[])` at 21444). The inset's clip rect and depth window already ride in camera slots 18-23, so C++ needs nothing new for it. The inset's kids stay on `triangles3d` (section 3, phase 3). + +Budget check: indoor world ≈ pipe 36 + wall 37 + hall 91 + water 153 + ~5 city blocks 145 + loop 256 = ~720 quads ≈ 1440 tris per view; two views ≈ 2900 of 8192 before fighters, kids (≤6000 cap on the inset buffer alone!) and HUD. `sceneDraw`'s return value should be summed and logged; also make `AC_NATIVE_FRAME`'s `trianglesDropped` log periodically, not once. + +## 3. Instancing + +Repeated geometry: skateboard deck (P1, P2, loose `ball.type==="skateboard"` boards via `drawBall` 18784), trucks (2 per board), wheels (4 per board), monowheel, city blocks (10, only 4 heights/colours), roof panes (20 identical), funboxes (6 identical), boost-pad chevrons (6 × 6), chain links, parkKids (8, but they are articulated — not instances of a mesh). + +Does `instances(handle, Float32Array transforms)` pay? The `emitTriangle` note (2145-2150) says twelve typed-array writes per face were dearer than twelve C-call args; a 16-float matrix per instance is the same trap. So: + +- **Rigid props: `meshDrawAt(handle, camera, x,y,z, yaw,pitch,roll, scale, r,g,b)` — 11 positional args, one call per instance.** A board is 3 handles (deck, truck, wheel) → 1 + 2 + 4 = 7 calls per board instead of ~30 `worldQuad` + 24 `worldCapsule` + 16 `disc3d` + 8 tris (~280 JS projections and ~110 native calls). C++ builds the rotation (yaw about y, pitch about z, roll about x — match `spunSkateFrame`/`skateFrame` order; verify against 18716 before wiring) and runs the retained-mesh path. Cracks (18748-18753) are per-board and rare: bake into a per-board deck handle when `boardCondition` changes. Wheel spin marks (18700-18705): draw the wheel handle with `roll = spin` so the marks rotate; the near/far shade difference (`side===nearSide`) becomes the tint args. +- **Static repeats (panes, cubes, blocks, chevrons): do not instance.** They are already baked into per-map handles; instancing only saves upload memory, which is nothing here. +- **Kids: no mesh instancing — but share the segment list.** Both the main view (`drawRunner`) and the inset (`rebuildSeatInset` 21482-21492) need every kid's `runnerWorldGeometry` segments as world-space capsules. A native `segments3d(buffer, count, camera)` with 11 floats/segment `[x1,y1,z1,x2,y2,z2,worldWidth,r,g,b,depthBias]` lets JS fill one buffer per kid per frame (or per 30 Hz tick) and draw it with two cameras. Typed-array writes cost, but 11 writes per segment × ~20 segments × 9 bodies ≈ 2000 writes vs. today's inset path (4 object literals + 2 `local()` allocs + clip + 12 pushes per segment, then a whole-array `Float32Array` copy). If writes still hurt, C++ can read a plain JS array of numbers via `JS_GetPropertyUint32` (slower per element than a typed array but avoids the `values.push` + copy); measure both. This overlaps the character-side researcher's lane — coordinate the buffer layout with them so `drawRunner` and the inset use the same fill. +- **Native-side instance lists with deltas**: only worth it for kids (positions change every frame anyway) — no. For loose boards (rare, ≤3) per-call `meshDrawAt` is fine. + +## 4. HUD and text + +Cost split: HUD boxes/lines are 2 `triangle3d` each at `hudDepth` (cheap; `drawStatusPiano` is 12 tris; the debug frame meter is the only heavy one and is debug-only). Text is where the cost is, and it is in two places: +- JS: string formatting per frame (finding 9), `handleWidth` (15775: `[...handle].reduce` allocates per call), `typeWrite` → `comicWrite` marshals a C string per call. +- Render side (`renderCpuMs`, not the 24 ms): `drawPackagedFont` (App.cpp 2618-2655) recomputes glyph indices and metrics for every string every frame, and `drawPlayerHandle` submits one string per glyph per pass. `kMaxSystemDraws = 128` drops the rest silently (`systemDropped`). + +Retained HUD widgets (native widgets updated by value) are **not worth it**: boxes are already on the triangle pass at the right depth (2178-2201 explains why), and a widget tree would just move the string formatting. Do instead: +1. Restore the 10 Hz string cache for the seat HUD: build `measure/verb/item` into `player.hudText` every 100 ms (the pattern at 5284-5296), and format `fps` at 2 Hz. Saves the per-frame `Math.round(...)+' bpm'` churn and the per-frame C-string marshal — ~0.2-0.3 ms JS. +2. Native glyph-run cache keyed `(family, emSize, text)` with a small LRU in `drawPackagedFont`, so a stable string costs one `DrawGlyphRun` and no metrics lookups. Render-side win only. +3. `comicWriteRun(text, x, y, size, Float32Array rgbPerGlyph)` (or `Uint8Array`) for nameplates so `drawPlayerHandle` submits one draw per pass instead of one per glyph — halves system draws and removes the 128-cap risk in fights. Optional; freeskate does not draw nameplates. +4. Keep `box`/`line` for HUD off the console path (they land under the world); nothing to change — `hudBox`/`hudLine` already do this. + +## 5. Phased plan + +Web shell rule for every phase: all native names are gated by `typeof x === "function"`; the JS branches that exist today (`drawQuadMesh` fallback 21190-21212, `rebuildSeatInset` fallback 21463-21473, `drawSkateboard` as is) remain the web/Mac path. `xbox/live/mac-test.html` (578) defines only `triangle`; `macos-native/main.swift` (802, 877) defines only `triangle3d`; `scene3d.mjs`/`scene3d-webgl.mjs` are a triangle sink with the same `(z+1.5)/3` depth mapping and need no change. No WebGL twin is required for correctness; one could later implement `meshUpload/meshDraw` in JS on top of `OskiewarScene3D.triangle` if the web ever wants the same call shape. + +### Phase 1 — one day, shippable, ~2.5 ms + +JS/C++ API: `meshUpload(vertices, faces, capsules) → handle`, `meshDraw(handle, camera, r?, g?, b?)`, `meshFree(handle)` in `ScenePrimitives.inc`, registered next to `sceneMesh` (QuickJsEngine.cpp 971). Keep `sceneMesh` for one release as the fallback when upload fails (returns −1). + +JS changes: +- `captureQuadMesh` 21172: emit capsules as a flat 11-float array alongside `mesh.capsules`; compute `mesh.bounds` at capture (fixes finding 2 for the fallback too). +- `nativeMeshBuffers` 21504: add `mesh.nativeHandle = meshUpload(vertices, faces, capsules)`; on rebuild of a keyed mesh, `meshFree` the old handle (`pipeSurfaceKey`, `pipeWallKey`, `hallMeshKey`, `waterMeshKey`, `loop.meshKey`, `cityBlocks`). +- `drawQuadMesh` 21186: native branch becomes `meshDraw(mesh.nativeHandle, mainNativeCamera())` — no capsule loop. +- `mainNativeCamera` 21508 and the inset camera 21459: write into two preallocated `Float32Array(27)`s. +- `rebuildSeatInset` 21444-21461: `meshDraw(handle, insetCamera)` per mesh; the C++ AABB cull replaces the dead JS bounds test; skip city blocks outside `[eye.x − 3500, eye.x + 3500]` in JS (cheap pre-filter). +- `drawIndoorHall` 757: draw the retained hall even while `roofShards.length`, and draw only the shard loop (831-838) in JS. + +C++: per-handle validation and AABB at upload; pre-lit colours; stack `ScenePoint[9]` clipping; all-inside fast path. + +Expected: capsule replay −1.0..−1.5 ms, inset uncullled re-sends and allocations −0.5, C++ alloc churn −0.3, roof-shard spike gone. Measure with the existing per-function laps and `AC_NATIVE_PROFILE hostJsMs`. + +### Phase 2 — two to three days, ~3.5 ms + +API: `sceneDraw(camera, viewMask)`, `meshFlags(handle, flags)`, `meshDrawAt(handle, camera, x,y,z, yaw,pitch,roll, scale, r,g,b)`. + +- Replace the 13 `meshDraw` calls in the main view and the inset loop with `sceneDraw(mainCamera,1)` at 20384 and `sceneDraw(insetCamera,2)` at 21444 (−0.3 ms of call overhead and JS loop; the cull is already native from phase 1). +- Skateboard as three handles (deck per board when cracks change; one shared truck; one shared wheel) drawn with `meshDrawAt` in `drawSkateboard`/`drawBoardRunningGear`/`drawTruck`; monowheel as one handle. −1.5 ms (`drawSkateboard` 1.9 → ~0.4 with 2-3 boards). +- Outdoor course: upload the thinned `terrainDrawProfiles` (17976) as chunked strip handles per LOD (surface/front/back, ~8 tiles per chunk, AABB each) and pick the LOD tier in JS as `activeTerrainProfile` does (17987); indoor is already retained. Small in the park; matters on the 320-tile course and in wide shots. +- Ponds/boost chevrons/ledges on the outdoor course: capture with `captureQuadMesh` like the loops (they are static; today they are re-projected per frame at 1457-1486). −0.3 ms outdoors. +- Decals: `meshAppend(handle, vertices, faces)` for an append-only per-map decal handle; blood/ding fans stay JS (they need the affine screen placement). −0.3 ms in long rounds. +- Log `sceneDraw`'s emitted count and periodic `trianglesDropped`. + +### Phase 3 — kids, text, polish, ~2 ms + +API: `segments3d(buffer, count, camera)` (11 floats/segment), `comicWriteRun(text, x, y, size, colors)`, native glyph-run cache. + +- Inset kids: fill one segment buffer per body (shared with the main-view character path if the character lane adopts the same layout) and draw it with the inset camera; drop the 30 Hz `values`/`Float32Array` rebuild and `triangles3d` replay. Heads stay as `disc3d` with the inset projection or become 3 segments each. −1.5 ms of the inset's 4.2. +- Seat HUD strings at 10 Hz, FPS at 2 Hz; `handleWidth` without the spread. −0.3 ms JS. +- Glyph-run cache and per-run coloured `comicWriteRun`: render-side only, but it also lifts the 128-draw cap risk in fights. +- Optional tint use: booster-pad pulse and the hall's `hallAir` fog become tint args instead of colour rebuilds. + +Cumulative world-side estimate: ~8 ms of the 24 (phase 1 ≈ 2.5, phase 2 ≈ 3.5, phase 3 ≈ 2). Combined with the character lane this is the path to <16 ms; the retained scene alone does not get there because the fighters, kids and text are the other half of the frame.