# GPU-Side Optimization Opportunities in OpenMW
This document analyzes the OpenMW graphics pipeline, scene graph traversal, and GLSL shaders to identify high-impact GPU-side optimization opportunities.
### Empirical Optimization Scorecard (Hardware Benchmark on Apple Silicon)
##### 1. Tested & Successfully Deployed Optimizations
| Commit | Optimization | Subsystem / Mechanism | Measured Impact (Hardware Timedemo) | Status |
| :--- | :--- | :--- | :--- | :---: |
| [`dee26420f3`](file:///Users/stellz/repos/openmw) | **Progressive Cascade Small-Feature Culling** | [`MWShadowTechnique`](file:///Users/stellz/repos/openmw/components/sceneutil/mwshadowtechnique.cpp): Fixed uninitialized pixel size vector; added progressive per-cascade culling (`12px, 50px, 120px, 300px`). | **+16.3% FPS** (+7.2 FPS)
**-4.66 ms GPU time** (-15.3%)
**307 draw calls eliminated/frame** | **Verified & Committed** |
| [`87ad857410`](file:///Users/stellz/repos/openmw) | **Resolution-Scaled Camera Small-Feature Culling** | [`RenderingManager`](file:///Users/stellz/repos/openmw/apps/openmw/mwrender/renderingmanager.cpp): Replaced static 2.0px threshold with resolution-scaled culling relative to 1080p reference height. | **+18.4% FPS** (+9.2 FPS)
**-3.60 ms GPU time** (-13.5%)
**-3.50 ms Draw Traversal** (-17.9%) | **Verified & Committed** |
| [`6426c62780`](file:///Users/stellz/repos/openmw) | **Interior Static Architecture Batching** | [`Scene::preload`](file:///Users/stellz/repos/openmw/apps/openmw/mwworld/scene.cpp): Batch interior walls/floors into unified static meshes via `StaticBatchingVisitor`. | **66% static node reduction** in interior cells; consolidated draw submission; performance neutral. | **Verified & Committed** |
| [`6d8a54cce2`](file:///Users/stellz/repos/openmw) | **Invariant Hoisting & Depth Pruning in `fog.glsl`** | [`fog.glsl`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/fog.glsl): Hoisted Z-row dot product to eliminate 4x4 matrix multiply per fragment; folded depth correction math into compile-time constants. | **+8.4% FPS** (+4.5 FPS)
**-1.59 ms GPU time** (-6.4%)
**-1.45 ms Draw Traversal** | **Verified & Committed** |
| [`ff6bfa4657`](file:///Users/stellz/repos/openmw) | **Single-Pass Multi-Texture Terrain Splatting** | [`material.cpp`](file:///Users/stellz/repos/openmw/components/terrain/material.cpp), [`terrain.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/terrain.frag): Consolidated 2–6 multi-pass layers into a single opaque `RenderBin 0` pass; evaluated lighting, 4-cascade shadows, and fog once per pixel. | **+7.5% FPS** in Pelagiad (+5.4 FPS)
**-0.96 ms frame time**
**-0.56 ms GPU time**
**50%–75% terrain draw calls eliminated** | **Verified & Committed** |
| [`c56f296a98`](file:///Users/stellz/repos/openmw) | **Linear Depth Cascade Split Early-Out & Shadow Distance Culling Bugfix** | [`shadows_fragment.glsl`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/shadows_fragment.glsl), [`mwshadowtechnique.cpp`](file:///Users/stellz/repos/openmw/components/sceneutil/mwshadowtechnique.cpp): Passed `uniform vec4 cascadeSplits;` from cull visitor to fragment shader to skip light-space division/bounds tests for earlier cascades when `distance > cascadeFar`. Fixed distance culling in `terrain.frag`, `groundcover.frag`, `water.frag`, and `bs/default.frag` by passing view distance `-passViewPos.z` instead of clip-space `linearDepth`. | **-1.11 to -1.53 ms GPU time** in Pelagiad (-7% to -8.6%)
**Peak 79.8 FPS** (+2.4 FPS)
Neutral in Balmora (57.9 vs 58.1 FPS, 22.45 vs 22.50 ms GPU) | **Verified & Committed** |
| [`7bb3541b65`](file:///Users/stellz/repos/openmw) | **Shared Shadow Receiver StateSet & RenderStage Hoisting** | [`MWShadowTechnique`](file:///Users/stellz/repos/openmw/components/sceneutil/mwshadowtechnique.cpp): Attached `shadowReceiverStateSet` directly to `RenderStage` with `OVERRIDE` flags instead of pushing to CullVisitor stack; pruned intermediate `StateGraph` level across thousands of drawables. | **-0.4 ms Draw Traversal** across both scenes (peak -1.18 ms in Balmora); **-1.02 ms GPU time** in Pelagiad (-5.7%); peak 58.6 FPS in Balmora, 77.7 FPS in Pelagiad. | **Verified & Committed** |
| [`d1bd976ec6`](file:///Users/stellz/repos/openmw) | **Material Uniform Deduplication & Shared StateSet Caching** | [`material.hpp`](file:///Users/stellz/repos/openmw/components/sceneutil/material.hpp), [`material.cpp`](file:///Users/stellz/repos/openmw/components/sceneutil/material.cpp): Cached and shared 8 canonical `osg::Uniform` instances per `Material` attribute instead of allocating new uniform heap objects for every StateSet; enabled uniform pointer equality matching in OSG `StateSet::compare`. | **Peak 60.4 FPS in Balmora** (+2.1 FPS, avg 58.7 FPS)
**-0.45 ms Draw Traversal** (down to 15.35 ms)
**-0.48 ms GPU time** (down to 21.72 ms)
78.4 FPS in Pelagiad
Eliminated thousands of dynamic heap allocations/cell | **Verified & Committed** |
| [`baa9fe40d8`](file:///Users/stellz/repos/openmw) | **Zero-Weight Diffuse Fetch Elimination in Single-Pass Terrain** (Opportunity #23) | [`terrain.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/terrain.frag): Sample low-resolution `blendMap1..5` splat alpha masks first; guard high-resolution `diffuseMap1..5` texture fetches with `if (w > 0.001)`. For localized textures where secondary layers cover <30% of a chunk, skips millions of wasted high-res texture fetches. | **+2.29% FPS** in Pelagiad (+1.71 FPS, 76.42 vs 74.71 FPS baseline)
**-0.30 ms frame time** (13.09 vs 13.39 ms)
Frametime StDev reduced by **40%** (±1.39 vs ±2.32 ms)
**+1.42% FPS** in Balmora (+0.80 FPS, 56.94 vs 56.14 FPS)
**-0.27 ms GPU time** | **Verified & Committed** |
#### 2. Tested & Failed / Rejected Hypotheses (Post-Mortem Log)
| Hypothesis / Opportunity | Expected Gain | Measured Hardware Result | Architectural / Hardware Post-Mortem | Final Status |
| :--- | :---: | :---: | :--- | :---: |
| **Out-of-Volume Shadow Map Sampling Pruning** (Opportunity #25) | +0.5% – 1.5% | **-1.89 FPS in Pelagiad** (74.53 vs 76.42 FPS)
**-1.03 FPS in Balmora** (55.91 vs 56.94 FPS)
Net regression | **Cascade Split Distance Already Partitions Depth:** Prior split distance screening (`distance <= cascadeSplits[...]`) already selects the appropriate cascade along the view ray. Visible fragments within the 2D footprint $[0, 1]^2$ almost always lie within the $[0, 1]$ depth range. Extending to a 3D vector comparison added ALU instruction overhead on every fragment across the screen without pruning meaningful fetches. | **REJECTED / REVERTED** |
| **Rasterizer Interpolator Pruning (`normalToViewMatrix` Elimination)** (Opportunity #20) | +1% – 3% | **-2.00 FPS in Pelagiad** (74.42 vs 76.42 FPS)
**-2.39 FPS in Balmora** (54.55 vs 56.94 FPS)
Net regression | **Driver Cross-Stage Dead Code Elimination (DCE):** Modern GPU compilers automatically strip unreferenced vertex-to-fragment varyings at pipeline link time. `normalToViewMatrix` was already unread in `objects.frag`, `terrain.frag`, and `bs/default.frag`. In `groundcover.frag`, replacing it with `normalize(gl_NormalMatrix * passNormal)` forced dynamic uniform loading and matrix-vector math into the fragment ALU for millions of grass fragments. | **REJECTED / REVERTED** |
| **Pre-Fetch Distance Cutoff Discard in `groundcover.frag`** (Opportunity #24) | +1% – 2% | **-2.06 FPS in Pelagiad** (74.36 vs 76.42 FPS)
**-1.27 FPS in Balmora** (55.67 vs 56.94 FPS)
Net regression | **Triangles Already Culled in Vertex Stage:** In `groundcover.vert`, triangles beyond fade distance are already collapsed to zero-area points `(0,0,0,1)` and culled by the hardware rasterizer before fragment generation. Adding an explicit `discard` check in the fragment stage only introduced dynamic branch divergence into valid foreground quads. | **REJECTED / REVERTED** |
| **StateSet Flattening & Hierarchy Consolidation** (Opportunity #19) | +5% – 10% | **+196 draw calls in Balmora (+6.6%)**
**+128 draw calls in Pelagiad (+6.1%)**
Draw Calls: 3,179 vs 2,983 (Balmora), 2,237 vs 2,109 (Pelagiad)
Framerate: 60.5 vs 60.4 FPS (Balmora), 77.7 vs 77.4 FPS (Pelagiad) | **Fractured Exterior Geometry Batches:** OpenMW's object paging engine relies on `MergeGeometryVisitor` to merge static objects (walls, rocks, paths) across chunks. `LessGeometry` groups candidate geometries strictly by raw `osg::StateSet*` pointer equality (`lhs->getStateSet() < rhs->getStateSet()`). In stock OpenMW, static leaf geometries share `nullptr` StateSets, matching and merging into giant unified draw calls. Pushing StateSets down to leaf geometries in `SceneManager` caused each template to receive distinct heap-allocated StateSet pointers. `LessGeometry` saw mismatched pointers across different objects and failed to merge them, fracturing chunk batches into hundreds of extra draw calls. Fixing this requires a global StateSet value canonicalizer across all paged cells—an enormous risk/complexity burden for virtually zero real-world FPS benefit. | **REJECTED / ABANDONED** |
| **Universal Directional Light & Shader Math Invariant Hoisting** (Opportunity #14) | +3% – 6% | **0.0% FPS gain**
Pelagiad: 76.6 vs 76.5 FPS
Balmora: 56.8 vs 56.2 FPS
Pure noise | **ALU Savings Masked by Memory Latency:** Pre-normalizing sun direction on CPU, guarding dead screen-space division, and hoisting vertex shadow offset saved minor scalar ALU instructions. On modern TBDR and desktop GPUs, simple math is completely latency-hidden behind texture sampling and memory fetch stalls. Variations in GPU timer and Draw Traversal were purely thermal/system noise (< 0.1 ms delta). | **REJECTED / REVERTED** |
| **Front-to-Back Sorting for `RenderBin 0`** (Opportunity #11) | +8% – 18% | **-11.0% FPS** (44.6 vs 50.1 FPS)
Severe regression | **State Thrashing & Redundant on TBDR:** Sorting drawables by distance broke OSG's `SORT_BY_STATE`, causing massive shader, texture, and uniform thrashing. Furthermore, Apple Silicon's Tile-Based Deferred Renderer (TBDR) hardware already does on-chip Hidden Surface Removal (HSR) in tile memory before shading, making software front-to-back sorting completely counterproductive. | **REJECTED / ELIMINATED** |
| **Depth-Only Shaders for Opaque Shadow Casters** (Opportunity #2) | +10% – 20% | **+0.4% FPS** (50.3 vs 50.1 FPS)
Pure noise | **Driver Already Performs Depth-Invariant Fast-Z:** The Apple Silicon OpenGL-to-Metal driver already analyzes [`shadowcasting.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/shadowcasting.frag) and identifies that opaque casters have no color targets and no `discard`, tagging the pipeline as depth-invariant. The rasterizer was already running hardware Double-Speed Z. Separate programs added CPU state complexity with no GPU gain. | **REJECTED / ELIMINATED** |
| **Backface ($N \cdot L \le 0$) Sun Shadow Skip** (Opportunity #9) | +5% – 10% | **-1.1% FPS** (49.5 vs 50.1 FPS)
Slight regression | **SIMD Warp Divergence:** GPUs execute fragments in 32-thread SIMD warps. Because neighboring pixels across geometric curves and normal-mapped bumps face different directions, threads diverge and the GPU must execute both the branch and the shadow fetch anyway. The branch overhead and pipeline serialization hurt throughput. | **REJECTED / ELIMINATED** |
| **Placed Item / Clutter Batching** (`REC_MISC`, `REC_BOOK`) | +5% – 10% | **0.0% FPS gain** | **Broke View Frustum Culling:** Merging thousands of small items into a cell-wide static mesh prevented OSG from culling items behind the camera, increasing vertex submission and rendering off-screen geometry with zero net draw call savings. Reverted to preserve culling. | **REJECTED / REVERTED** |
| **Early Discard for Zero-Weight Terrain Passes** (Opportunity #10) | +5% – 12% | **+0.2% in Balmora** (57.66 vs 57.55 FPS)
**+0.8% in Pelagiad** (72.60 vs 72.00 FPS)
Minimal return | **Does Not Eliminate Draw Calls & SIMD Warp Divergence:** Multi-pass splatting still pays the CPU and vertex overhead of issuing 2–4 draw calls per chunk. On TBDR GPUs, `discard` causes quad divergence along soft texture blend transitions and disables tile fast-paths. True terrain performance requires Single-Pass Multi-Texturing (Opportunity #1). | **REJECTED / REVERTED** |
| **Shadow Caster Draw Call Elimination** (Blanket Shadow Cut) | +15% – 30% | **0.0% FPS gain in Balmora**
58.5 FPS (shadows on, 2,983 calls) vs 57.7 FPS (shadows off, 1,645 calls)
GPU: 22.44 vs 21.88 ms
Draw: 16.04 vs 16.35 ms | **StateGraph Asymmetry & CPU Bottleneck:** Cutting 1,338 shadow draw calls (-45%) only reduced OSG StateGraphs by 27 (536 vs 509). Shadow passes are depth-only with zero complex state or lighting math, taking <0.7 ms GPU time and <0.5 ms CPU draw overhead. The CPU render thread is 100% pinned by the 1,645 Main Camera draw calls and their 509 StateGraphs (~16.3 ms). | **REJECTED / ARCHITECTURAL FINDING** |
| **Water Reflection RTT Pass Frustum Culling** (Opportunity #3) | +15% – 35% | **N/A** (0% in benchmark) | **Feature Inactive in User Settings:** In default OpenMW settings, `[Water] shader = false`, meaning water reflection RTT passes are not even running in the active benchmark scenario. | **ON HOLD (Inactive)** |
#### 3. Untried / Remaining Optimization Opportunities (Active Backlog)
| Opportunity | Focus Area | Subsystem / Mechanism | Expected Impact | Implementation Complexity | Priority / Recommendation |
| :--- | :--- | :--- | :--- | :---: | :---: |
| **Early-Z and Overdraw Mitigation for Alpha-Tested Foliage** (Opportunity #4) | **Overdraw Mitigation & RenderBin Ordering** | [`groundcover.cpp`](file:///Users/stellz/repos/openmw/apps/openmw/mwrender/groundcover.cpp): Move groundcover from `RenderBin 0` (shared with opaque terrain/buildings) to `RenderBin 1`. Guarantees all opaque terrain and architecture render first and populate depth, enabling hardware Early-Z to instantly reject occluded grass quads before fragment shader execution. | **10% – 25%** in foliage / dense groundcover | **Very Low – Low** | **High Priority (Top Candidate)** |
| **Dead Math & Redundant Normalization Elimination in `objects.frag`** (Opportunity #21) | **Fragment ALU & Redundant Ops** | [`objects.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/objects.frag): Guard dead screen-space division (`screenCoords`), dead 3D normalization (`viewVec = normalize(passViewPos)`), and dead viewNormal transformations when features are inactive. | **-1% – 2% GPU fragment time** | **Very Low** | **Quick Win Candidate** |
| **Vertex Shadow Offset Hoisting in `shadows_vertex.glsl`** (Opportunity #22) | **Vertex Shader ALU** | [`shadows_vertex.glsl`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/shadows_vertex.glsl): Hoist `shadowOffset = vec4(viewNormal * @shadowNormalOffset, 0.0);` and `viewPos + shadowOffset` outside the unrolled 4-cascade loop in `setupShadowCoords(...)`. | **<1% Vertex ALU** | **Very Low** | **Low Priority** |
| **Water Wave Normal Texture Latency Decoupling** (Opportunity #8) | **Shader Optimization** | [`water.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/water.frag): Decouple 6 serialized/dependent normal map texture fetches into independent parallel UV lookups to hide memory latency on TBDR GPUs. | **3% – 7%** in water / coastal viewpoints | **Low** | **Medium Priority** (Inactive unless `shader = true`) |
| **Defaulting to Clustered Lighting (SSBO)** (Opportunity #5) | **Lighting Pipeline** | [`lighting_fragment_impl.glsl`](file:///Users/stellz/repos/openmw/files/shaders/lib/core/lighting_fragment_impl.glsl): Default to clustered forward shading compute grid over legacy unrolled uniform loops when multiple local point lights are active. | **10% – 20%** in dungeons and night settlements | **Low** (Config/Tuning) | **Medium Priority** |
| **Optimizing Cascaded Shadow Mapping: View-Depth Selection vs. Per-Cascade Varyings** (Opportunity #7) | **Interpolator & Register Pressure** | [`shadows_vertex.glsl`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/shadows_vertex.glsl), [`shadows_fragment.glsl`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/shadows_fragment.glsl): Select active cascade in fragment shader from view depth; eliminate per-cascade vertex varyings (4–8 vec4 registers freed across all shaders). | **3% – 8% GPU frame time** | **Medium** | **Architectural Project** |
| **Hardware GPU Vertex Skinning** (Opportunity #6) | **CPU/VBO Upload Elimination** | [`scenemanager.cpp`](file:///Users/stellz/repos/openmw/components/resource/scenemanager.cpp), vertex shaders: Move skeletal animation vertex transformation from CPU (software skinning + dynamic VBO uploads) to vertex shaders via matrix palettes or bone texture buffers. | **5% – 15%** in NPC-heavy towns (Balmora, Vivec) | **Medium – High** | **Major Architectural Project** |
| **Frustum-Culling the Water Reflection RTT Pass** (Opportunity #3) | **Pass Elimination** | [`water.cpp`](file:///Users/stellz/repos/openmw/components/terrain/world.cpp): Skip reflection camera render pass entirely when the water plane is outside the camera frustum or occluded. | **15% – 35%** (Only when water shaders enabled) | **Low – Medium** | **On Hold** (`[Water] shader = false` by default) |
---
## 1. [COMPLETED] Single-Pass Multi-Texture Terrain Splatting
> [!NOTE]
> **Status: Completed & Verified (Commit [`ff6bfa4657`](file:///Users/stellz/repos/openmw))**
> **Empirical Result:** **+7.50% FPS** in Pelagiad (+5.4 FPS, 77.40 vs 72.00 FPS), **-0.96 ms frame time** (-6.9%), **-0.56 ms GPU time**, **-0.42 ms Draw Traversal**. Neutral in Balmora (56.9 – 59.4 FPS vs 57.55 FPS).
> **Implemented Changes:** Consolidated multi-pass terrain splatting into a single opaque `RenderBin 0` pass for chunks with 2 to 6 texture layers; pre-cached uniforms in `UniformCollection`; evaluated blend weighting, 4-cascade shadows, lighting, and fog once per pixel in [`terrain.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/terrain.frag); disabled framebuffer blending.
### Overview & Problem
OpenMW implements terrain texture splatting as a multi-pass pipeline. In [`Terrain::Material::createPasses`](file:///Users/stellz/repos/openmw/components/terrain/material.cpp#L191-L274):
```cpp
for (std::vector::const_iterator it = layers.begin(); it != layers.end(); ++it)
{
// ...
stateset->setRenderBinDetails(firstLayer ? 0 : 1, "RenderBin");
if (!firstLayer)
{
stateset->setAttributeAndModes(BlendFunc::value(), osg::StateAttribute::ON);
stateset->setAttributeAndModes(EqualDepth::value(), osg::StateAttribute::ON);
}
// ...
stateset->setAttributeAndModes(shaderManager.getProgram("terrain", defineMap));
passes.push_back(stateset);
}
```
Every texture layer emits an independent rendering pass over the terrain chunk geometry. In every single pass, [`terrain.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/terrain.frag#L85-L107) executes:
```glsl
float shadowing = unshadowedLightRatio(linearDepth);
vec3 diffuseLight, ambientLight, specularLight;
doLighting(gl_FragCoord.xy, passViewPos, viewNormal, shininess, shadowing, diffuseLight, ambientLight, specularLight);
lighting = diffuseColor.xyz * diffuseLight + getAmbientColor(...) + getEmissionColor(...);
gl_FragData[0].xyz = gl_FragData[0].xyz * lighting + specular;
gl_FragData[0] = applyFogAtDist(gl_FragData[0], passViewPos, euclideanDepth, linearDepth, near, far);
```
If a terrain chunk blends 3 or 4 texture layers (standard throughout Morrowind):
- The GPU transforms and rasterizes the geometry 3 to 4 times.
- Shadow map projection and filtering ([`unshadowedLightRatio`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/shadows_fragment.glsl#L16)), dynamic lighting calculation ([`doLighting`](file:///Users/stellz/repos/openmw/files/shaders/lib/core/lighting_fragment_impl.glsl#L17)), and fog are recalculated 3 to 4 times for the exact same screen pixels.
### Implemented Solution
Consolidated terrain rendering into a single-pass shader:
1. Bind active diffuse maps and blend maps to consecutive texture units in a single `RenderBin 0` StateSet.
2. In [`terrain.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/terrain.frag), sample and blend active diffuse maps across active layers using normalized blend weights to produce a single final diffuse color.
3. Compute lighting, shadow map filtering across all 4 cascades, and fog **once** per pixel.
4. Disable hardware blending (`GL_BLEND` OFF) and enable standard depth writes (`LequalDepth`).
5. Maintain multi-pass fallback for composite map generation, chunks with $>6$ layers, or modded terrain normal maps.
### Performance Benefit Estimation & Empirical Results
- **Status:** **Completed & Verified (`ff6bfa4657`)**
- **Subsystem / Pass Reduction:** **50% – 75% reduction** in terrain draw calls and geometry submission.
- **Empirical Hardware Test Results (Apple Silicon, 3-Trial Averages):**
- **Pelagiad (`--timedemo=foliage` — >60% Terrain Coverage):**
- Framerate: **77.40 FPS** vs **72.00 FPS** baseline (**+7.50% FPS / +5.40 FPS throughput gain**)
- Frame Time: **12.92 ms** vs **13.88 ms** (**-0.96 ms reduction**)
- GPU Timer: **17.69 ms** vs **18.25 ms** (**-0.56 ms reduction**)
- Draw Traversal: **10.93 ms** vs **11.35 ms** (**-0.42 ms reduction**)
- **Balmora (`--timedemo=balmora` — Heavy Architecture):**
- Framerate: **56.9 – 59.4 FPS** vs **57.55 FPS** baseline (neutral; terrain is mostly occluded by buildings)
- GPU Timer: Peak runs dropped to **22.03 ms** (vs **23.40 ms** baseline, **-1.37 ms GPU savings**)
- **Bottlenecks Relieved:** Multi-pass overdraw, redundant vertex processing, 4-cascade shadow filtering latency, framebuffer blend stalls.
---
## 2. [FAILED / REJECTED] Depth-Only Programs for Opaque Shadow Casters (Unlocking Hardware "Double-Speed Z")
> [!CAUTION]
> **Status: Tested & Rejected (+0.4% FPS — Within Noise Margin)**
> **Empirical Result:** 50.3 FPS vs 50.1 FPS baseline in Balmora with heavy shadows.
> **Post-Mortem:** The Apple Silicon OpenGL-to-Metal driver already analyzes [`shadowcasting.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/shadowcasting.frag) and identifies that opaque casters have no color targets and no `discard`, tagging the pipeline as depth-invariant. The rasterizer was already running hardware Double-Speed Z. Separate programs added CPU state complexity with no GPU gain. **Do not re-attempt.**
### Overview & Problem
When generating shadow maps, OpenMW renders into a depth-only Framebuffer Object ([`_camera->attach(osg::Camera::DEPTH_BUFFER, _texture.get())`](file:///Users/stellz/repos/openmw/components/sceneutil/mwshadowtechnique.cpp#L621)), with no color attachment.
However, in [`SceneUtil::MWShadowTechnique::setupCastingShader`](file:///Users/stellz/repos/openmw/components/sceneutil/mwshadowtechnique.cpp#L966-L980), every shadow casting program binds both [`shadowcasting.vert`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/shadowcasting.vert) and [`shadowcasting.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/shadowcasting.frag). In [`SceneUtil::ShadowsBin::cullStateGraph`](file:///Users/stellz/repos/openmw/components/sceneutil/shadowsbin.cpp#L128-L136), opaque geometry (where `!state.needTexture()`) is sorted into `mNoTestStateSet`. This state set simply sets uniforms:
```cpp
mNoTestStateSet->addUniform(new osg::Uniform("useDiffuseMapForShadowAlpha", false));
mNoTestStateSet->addUniform(new osg::Uniform("alphaTestShadows", false));
```
It does **not** unbind the fragment shader. Because the root shadow state set forces `_castingPrograms` with `osg::StateAttribute::OVERRIDE`, the GPU executes [`shadowcasting.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/shadowcasting.frag) on **every opaque fragment across every shadow cascade**. In `shadowcasting.frag`:
```glsl
gl_FragData[0].rgb = vec3(1.0);
if (useDiffuseMapForShadowAlpha)
gl_FragData[0].a = texture2D(diffuseMap, diffuseMapUV).a * alphaPassthrough;
else
gl_FragData[0].a = alphaPassthrough;
gl_FragData[0].a = alphaTest(gl_FragData[0].a, alphaRef);
if (alphaTestShadows && gl_FragData[0].a <= 0.5)
discard;
```
### Proposed Solution
1. Create a depth-only `osg::Program` containing **only** [`shadowcasting.vert`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/shadowcasting.vert) (programs without fragment shaders are standard in OpenGL 2.0+ and core profiles for depth-only passes).
2. Set `glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE)` on shadow passes.
3. In [`SceneUtil::ShadowsBin`](file:///Users/stellz/repos/openmw/components/sceneutil/shadowsbin.cpp), bind the vertex-only program to all opaque shadow casters (`mNoTestStateSet`). Only assign programs with [`shadowcasting.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/shadowcasting.frag) to drawables that explicitly require alpha testing or alpha-to-coverage.
### Performance Benefit Estimation & Test Results
- **Status:** **Tested & Ineffective (+0.4% FPS — Within Noise Margin)**
- **Empirical Test Result:** Tested on hardware in Balmora with heavy shadows (50.3 FPS vs 50.1 FPS baseline).
- **Why It Failed:** The Apple Silicon OpenGL-to-Metal driver / compiler already detects that [`shadowcasting.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/shadowcasting.frag) contains no color writes or discards for opaque geometry and marks the pipeline state as depth-invariant. The hardware rasterizer was already running double-speed Z with zero fragment shading overhead. Splitting opaque casters into a separate vertex-only program added CPU state switching overhead with no measurable GPU gain.
---
## 3. Frustum-Culling the Water Reflection RTT Pass
### Overview & Problem
The water reflection node [`Reflection`](file:///Users/stellz/repos/openmw/apps/openmw/mwrender/water.cpp#L241) derives from [`SceneUtil::RTTNode`](file:///Users/stellz/repos/openmw/components/sceneutil/rtt.cpp#L22), which explicitly disables culling:
```cpp
setCullingActive(false);
```
In outdoor cells, whenever water exists anywhere within the loaded cell grid (even far off or entirely occluded by terrain/mountains), the `ReflectionCamera` executes every frame:
- Traverses the entire scene graph.
- Clips and renders terrain, statics, sky, and actors into a reflection render-target texture.
- This occurs even when the player is looking up at the sky, facing away from the water, or standing inside a mountain valley where no water is visible.
### Proposed Solution
1. Compute the bounding volume of the visible water surface in the active cells.
2. Frustum-cull the water surface against the primary camera prior to invoking the reflection camera cull traversal.
3. If the water plane is outside the camera frustum or occluded, skip the `ReflectionCamera` traversal and draw pass entirely for that frame.
### Performance Benefit Estimation
- **Subsystem / Pass Reduction:** **100% elimination** of the reflection render-to-texture pass whenever water is not in view.
- **Estimated End-to-End Frame Time Impact:** **15% – 35% overall GPU frame time reduction (~2.0 – 6.0 ms on a 60 FPS budget)** when facing away from water in exterior cells with water loaded.
- **Test Scenario:** Exterior world locations with water nearby while facing inland, looking toward hills, or looking at the sky.
- **Bottlenecks Relieved:** Draw-call volume, redundant scene geometry rasterization, render-target state transitions.
---
## 4. Early-Z and Overdraw Mitigation for Alpha-Tested Foliage & Groundcover
### Overview & Problem
OpenMW uses forward rendering. Because Alpha-to-Coverage is disabled unless MSAA is turned on ([`renderingmanager.cpp`](file:///Users/stellz/repos/openmw/apps/openmw/mwrender/renderingmanager.cpp)), all tree leaves, hair, cutouts, and grass blades invoke `discard` in [`alpha.glsl`](file:///Users/stellz/repos/openmw/files/shaders/lib/material/alpha.glsl#L61-L82).
Groundcover and foliage geometry are submitted into [`RenderBin 0`](file:///Users/stellz/repos/openmw/apps/openmw/mwrender/groundcover.cpp#L361) without front-to-back sorting. When a fragment shader uses `discard`, hardware Early-Z depth writes are disabled (or downgraded to conservative depth testing where writes cannot occur before shader completion).
In dense forests and high-density grass (e.g., Bitter Coast, Ascadian Isles, modded groundcover), dense overlapping quads cause 3x–6x pixel overdraw. Every occluded layer runs the heavy forward lighting shader ([`doLighting`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/groundcover.frag#L82), shadow map lookups, normal mapping) repeatedly before depth resolves.
### Proposed Solution
- **Shader Optimization:** Move the `alphaTest` / `discard` call to the absolute beginning of [`groundcover.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/groundcover.frag#L54-L63) and [`objects.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/objects.frag#L140-L165) before sampling normal maps, environment maps, or running shadow/lighting calculations.
- **Draw Ordering / Depth Prepass:** Ensure opaque geometry is rendered strictly before alpha-tested foliage, or introduce a lightweight depth prepass for alpha-tested geometry so that subsequent color passes achieve 100% early depth rejection.
- **Core GLSL:** For OpenGL 4.2+ targets, specify `layout(early_fragment_tests) in;` where applicable to discard occluded fragments before fragment execution.
### Performance Benefit Estimation
- **Subsystem / Pass Reduction:** **30% – 60% reduction** in fragment shading cost for foliage and groundcover.
- **Estimated End-to-End Frame Time Impact:** **10% – 25% overall GPU frame time reduction** in dense forests and high-density grass areas; negligible in barren terrain or interiors.
- **Test Scenario:** Bitter Coast swamps, dense Ascadian Isles tree groves, or exterior scenes with groundcover mods enabled.
- **Bottlenecks Relieved:** Fragment ALU overdraw, texture sampler bandwidth, shadow cascade filtering latency.
---
## 5. Defaulting to Clustered Lighting (SSBO) over Legacy Uniform Loops
### Overview & Problem
OpenMW ships with `clustered lighting = false` by default in [`settings-default.cfg`](file:///Users/stellz/repos/openmw/files/settings-default.cfg#L486).
In legacy mode, lighting uses an array of uniform matrices:
```glsl
uniform mat4 LightBuffer[@maxLights];
uniform int PointLightCount;
```
For each light, the shader extracts 8 distinct matrix components across 4 uniform vectors (`lcalcPosition`, `lcalcDiffuse`, `lcalcAmbient`, `lcalcConstantAttenuation`, etc.).
Furthermore, the CPU must compute and upload per-object light lists and uniform state sets on every draw call. In the fragment shader, `for (int i = 0; i < PointLightCount; ++i)` creates dynamic warp divergence across neighbouring pixels that receive differing numbers of lights.
### Proposed Solution
OpenMW already contains a compute-shader based clustered lighting implementation ([`cluster.comp`](file:///Users/stellz/repos/openmw/files/shaders/core/lighting/cluster.comp) and [`cull.comp`](file:///Users/stellz/repos/openmw/files/shaders/core/lighting/cull.comp)) that clusters point lights into a 3D frustum grid stored in an SSBO.
Promoting clustered lighting to default on modern platforms:
- Eliminates per-object uniform state updates.
- Restricts light evaluation to only the lights that touch each pixel's 3D cluster tile.
- Eliminates dynamic uniform matrix unpack overhead in the fragment shader.
### Performance Benefit Estimation
- **Subsystem / Pass Reduction:** **40% – 70% reduction** in point light shading cost in light-dense scenes.
- **Estimated End-to-End Frame Time Impact:** **10% – 20% overall GPU frame time reduction** in interior dungeons, caves, and nighttime settlements with many torches/lanterns; minimal difference in daytime exteriors where directional sun lighting dominates.
- **Test Scenario:** Complex interiors (e.g., Balmora Guild of Mages, Vivec cantons, ancestral tombs) or towns at night.
- **Bottlenecks Relieved:** Uniform register pressure, warp divergence, draw-call state changes.
---
## 6. Hardware GPU Vertex Skinning (Eliminating Per-Frame VBO Re-uploads)
### Overview & Problem
OpenMW currently performs skeletal animation skinning entirely on the CPU. In [`SceneUtil::RigGeometry::cull`](file:///Users/stellz/repos/openmw/components/sceneutil/riggeometry.cpp#L163-L228):
```cpp
// skinning
const osg::Vec3Array* positionSrc = static_cast(mSourceGeometry->getVertexArray());
const osg::Vec3Array* normalSrc = static_cast(mSourceGeometry->getNormalArray());
const osg::Vec4Array* tangentSrc = mSourceTangents;
osg::Vec3Array* positionDst = static_cast(geom.getVertexArray());
osg::Vec3Array* normalDst = static_cast(geom.getNormalArray());
osg::Vec4Array* tangentDst = static_cast(geom.getTexCoordArray(7));
// ... matrix multiplication loop over vertices ...
positionDst->dirty();
if (normalDst) normalDst->dirty();
if (tangentDst) tangentDst->dirty();
geom.osg::Drawable::dirtyGLObjects();
```
Every frame, for every visible actor, creature, and player, vertex positions, normals, and tangents are computed on the CPU and re-uploaded over the PCIe bus to dynamic VBOs (`GL_DYNAMIC_DRAW_ARB`).
### Proposed Solution
Implement GPU skinning:
1. Keep mesh vertex attributes (positions, normals, bone indices, bone weights) static in VRAM.
2. Upload bone transform matrices for the frame into a Uniform Buffer Object (UBO) or Shader Storage Buffer Object (SSBO).
3. Compute vertex and normal skinning in [`objects.vert`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/objects.vert) (or via a compute shader prepass).
### Performance Benefit Estimation
- **Subsystem / Pass Reduction:** **100% elimination** of skeletal vertex data transfers over PCIe; eliminates CPU skinning traversal.
- **Estimated End-to-End Frame Time Impact:** **5% – 15% frame time reduction** in actor-heavy scenes (Balmora, Vivec, large battles) by eliminating CPU-GPU driver synchronization stalls (buffer orphaning / wait-for-idle) and PCIe bus contention.
- **Test Scenario:** Populated settlements with 20–50 NPCs and guards in view.
- **Bottlenecks Relieved:** PCIe transfer bandwidth, driver buffer synchronization stalls, CPU cull-stage bottlenecks.
---
## 7. Optimizing Cascaded Shadow Mapping: View-Depth Selection vs. Per-Cascade Vertex Varyings
### Overview & Problem
In [`setupShadowCoords`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/shadows_vertex.glsl#L18-L46), the vertex shader iterates over every shadow texture unit:
```glsl
@foreach shadow_texture_unit_index @shadow_texture_unit_list
shadowRegionCoords@shadow_texture_unit_index = validRegionMatrix@shadow_texture_unit_index * viewPos;
shadowSpaceCoords@shadow_texture_unit_index = shadowSpaceMatrix@shadow_texture_unit_index * offsetViewPosition;
@endforeach
```
For a 4-cascade setup, this calculates 4 to 8 matrix-vector multiplications per vertex and exports 4 to 8 separate `varying vec4` interpolators. In [`shadows_fragment.glsl`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/shadows_fragment.glsl#L26-L43), the fragment shader performs perspective division on all of them and checks bounding box intervals inside an unrolled loop:
```glsl
vec3 shadowXYZ = shadowSpaceCoords@shadow_texture_unit_index.xyz / shadowSpaceCoords@shadow_texture_unit_index.w;
if (all(lessThan(shadowXYZ.xy, vec2(1.0, 1.0))) && all(greaterThan(shadowXYZ.xy, vec2(0.0, 0.0)))) {
shadowing = min(shadow2DProj(...).r, shadowing);
doneShadows = ...;
}
```
Hardware interpolators are strictly limited (typically 16 to 32 `vec4` registers). Consuming 4–8 varyings for shadows alone across all shaders increases interpolator bandwidth and limits GPU warp/wavefront occupancy. Furthermore, inactive cascades are transformed at every vertex regardless of whether that polygon lies in that cascade's frustum.
### Proposed Solution
1. In the fragment shader, determine the active cascade index using the already-computed view depth (`-passViewPos.z` or `linearDepth`) against split plane uniform thresholds.
2. Calculate light-space coordinates in the fragment shader for that specific cascade, or sample from a `sampler2DArrayShadow` using the selected slice.
3. Eliminate per-cascade vertex varyings, reducing interpolator bandwidth and vertex shader instructions across all geometry.
### Performance Benefit Estimation
- **Subsystem / Pass Reduction:** **15% – 25% reduction** in vertex shader ALU on shadow receivers; frees 4–8 varyings per vertex.
- **Estimated End-to-End Frame Time Impact:** **3% – 8% overall GPU frame time reduction** across all exterior scenes when shadows are enabled.
- **Test Scenario:** Complex geometric scenes with high vertex density (towns, ruins, dense terrain).
- **Bottlenecks Relieved:** Hardware interpolator register pressure, GPU warp/wavefront occupancy limits, vertex ALU.
---
## 8. Water Wave Normal Texture Latency Decoupling & Depth Sampling
### Overview & Problem
In [`water.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/water.frag#L113-L118):
```glsl
vec3 normal0 = 2.0 * texture2D(normalMap, normalCoords(UV, ..., vec3(0.0))).rgb - 1.0;
vec3 normal1 = 2.0 * texture2D(normalMap, normalCoords(UV, ..., normal0)).rgb - 1.0;
vec3 normal2 = 2.0 * texture2D(normalMap, normalCoords(UV, ..., normal1)).rgb - 1.0;
vec3 normal3 = 2.0 * texture2D(normalMap, normalCoords(UV, ..., normal2)).rgb - 1.0;
vec3 normal4 = 2.0 * texture2D(normalMap, normalCoords(UV, ..., normal3)).rgb - 1.0;
vec3 normal5 = 2.0 * texture2D(normalMap, normalCoords(UV, ..., normal4)).rgb - 1.0;
```
Each wave layer's UV offset depends directly on the RGB normal of the previous texture sample. This creates a **6-stage serialized dependent texture read chain**.
On modern GPUs, texture fetch latency is hidden by executing instructions from other warps or issuing multiple independent fetches simultaneously. A serial chain forces execution pipelines to stall at every single tap. Furthermore, [`hasOpaqueGeometry`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/water.frag#L87-L101) executes a 3x3 loop (9 texture fetches) into the depth buffer for every water fragment.
### Proposed Solution
1. Decouple normal map UV coordinates using independent scrolling vectors and frequencies so texture samplers can issue fetches in parallel.
2. Replace the 3x3 depth kernel in `hasOpaqueGeometry` with a single depth sample or a hierarchical/downscaled depth test.
### Performance Benefit Estimation
- **Subsystem / Pass Reduction:** **35% – 50% reduction** in `water.frag` execution time.
- **Estimated End-to-End Frame Time Impact:** **3% – 7% overall GPU frame time reduction** whenever open water occupies a substantial portion of the screen viewport.
- **Test Scenario:** Coastal areas, boat travel, swimming underwater looking up at the water plane.
- **Bottlenecks Relieved:** GPU memory latency stalls, texture cache bandwidth.
---
## 9. [FAILED / REJECTED] Backface & Shadow-Facing Light Skip ($N \cdot L \le 0$) in Shaders
> [!CAUTION]
> **Status: Tested & Rejected (-1.1% FPS Regression)**
> **Empirical Result:** 49.5 FPS vs 50.1 FPS baseline in Balmora timedemo.
> **Post-Mortem:** GPUs execute fragment shaders in 32-thread SIMD warps. Because neighboring pixels across geometric curves and normal-mapped bumps face different directions, threads diverge and the GPU must execute both the branch and the shadow fetch anyway. The branch overhead and pipeline serialization hurt throughput. **Do not re-attempt.**
### Overview & Problem
Directional sunlight lighting in OpenMW is computed in [`files/shaders/lib/core/lighting_fragment_impl.glsl`](file:///Users/stellz/repos/openmw/files/shaders/lib/core/lighting_fragment_impl.glsl#L25-L28):
```glsl
calcDirectionalLighting(sun, viewDir, viewNormal, shininess, diffuseLight, ambientLight, specularLight);
diffuseLight *= shadowing;
specularLight *= shadowing;
```
For directional sunlight, diffuse lighting follows Lambert's cosine law: $I_d = \max(\vec{n} \cdot \vec{l}, 0.0)$, and specular lighting similarly requires $\vec{n} \cdot \vec{l} > 0.0$ ([`files/shaders/lib/light/util.glsl`](file:///Users/stellz/repos/openmw/files/shaders/lib/light/util.glsl#L16-L43)).
When a surface normal faces away from the sun ($\vec{n} \cdot \vec{l} \le 0.0$), `diffuseLight` and `specularLight` are strictly `vec3(0.0)`. Multiplying `0.0 * shadowing` always equals `0.0`.
However, in [`objects.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/objects.frag#L218), [`terrain.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/terrain.frag#L85), and [`groundcover.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/groundcover.frag#L75), `unshadowedLightRatio()` is invoked unconditionally on every single fragment on screen.
Across every shadow-facing face of buildings, rocks, terrain hillsides, trees, and groundcover, the GPU executes light-space coordinate divisions, bounding checks, and PCF depth texture lookups across all 4 shadow cascades—even though the resulting shadow multiplier has zero mathematical effect on the pixel output.
### Proposed Solution
Wrap the shadow map evaluation in a dot product check against the normalized sun direction in view space:
```glsl
float shadowing = 1.0;
if (dot(viewNormal, normalize(sun.position.xyz)) > 0.0)
shadowing = unshadowedLightRatio(linearDepth);
```
*(For two-sided groundcover with translucent transmission, the threshold can be adjusted or two-sided illumination accounted for).*
### Performance Benefit & Empirical Benchmark
- **Theoretical Subsystem / Pass Reduction:** Up to 30% – 50% reduction in shadow map filtering on shadow-facing pixels.
- **Empirical Test Results (Apple M-series, Balmora Timedemo, 3-trial averages):**
- Control Baseline: **50.1 FPS** (19.98 ms frame time, 26.13 ms GPU timer)
- With Backface Shadow Skip: **49.5 FPS** (20.18 ms frame time, 26.75 ms GPU timer)
- **Net Result:** **No measurable performance gain** (-0.6 FPS / within noise threshold).
- **Architectural Bottleneck / Why It Fails:**
GPUs execute fragment shaders in SIMD execution warps (32 or 64 threads). Across complex meshes, normal-mapped surfaces, and terrain slopes, pixels within the same 32-thread warp face different directions. Because threads in a warp diverge, the GPU execution engine must execute *both* paths unless all 32 threads face away from the sun. The added dynamic branch instructions and disrupted texture fetch pipelining negate any potential ALU savings.
- **Implementation Complexity:** Extremely Low (1–2 lines of GLSL).
---
## 10. [FAILED / REJECTED] Early Discard for Zero-Weight Secondary Terrain Splat Passes
> [!CAUTION]
> **Status: Tested & Rejected (+0.2% in Balmora, +0.8% in Pelagiad — Minimal Return)**
> **Empirical Result:** 57.66 vs 57.55 FPS in Balmora (+0.2%); 72.60 vs 72.00 FPS in Pelagiad (+0.8%).
> **Post-Mortem:** Multi-pass splatting still pays the CPU and vertex overhead of issuing 2–4 draw calls per chunk. On TBDR GPUs, `discard` causes quad divergence along soft texture blend transitions and disables tile fast-paths. True terrain performance requires Single-Pass Multi-Texturing (Opportunity #1). **Do not re-attempt.**
### Overview & Problem
In multi-pass terrain splatting ([`Terrain::Material::createPasses`](file:///Users/stellz/repos/openmw/components/terrain/material.cpp#L191-L229)), terrain chunks blend 3 to 4 texture layers. The base layer (`firstLayer`) writes depth with `LequalDepth` and establishes base color.
Subsequent passes (`!firstLayer`) use additive blending (`BlendFunc(SRC_ALPHA, ONE)`) and `EqualDepth` testing with depth writes disabled.
In any given terrain chunk, blendmaps contain extensive areas where a layer's weight is `0.0` (because that particular texture only appears on a small patch of dirt, rock, or cobblestone).
Currently in [`terrain.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/terrain.frag#L69-L112):
```glsl
#if @blendMap
vec2 blendMapUV = (texMat1 * vec4(uv, 0.0, 1.0)).xy;
gl_FragData[0].a *= texture2D(blendMap, blendMapUV).a;
#endif
```
Even if `gl_FragData[0].a` evaluates to `0.0`, the shader proceeds to:
1. Fetch and reconstruct the normal map.
2. Execute `unshadowedLightRatio(linearDepth)` across all 4 shadow cascades.
3. Compute directional and point lighting via `doLighting()`.
4. Calculate atmospheric fog via `applyFogAtDist()`.
5. Write to color and normals MRTs (`gl_FragData[0]` and `gl_FragData[1]`).
All of this work is done only to multiply the final color by `0.0` and add `0.0` to the frame buffer!
### Proposed Solution
Add an immediate discard check right after reading the blend map on secondary passes:
```glsl
#if @blendMap
vec2 blendMapUV = (texMat1 * vec4(uv, 0.0, 1.0)).xy;
float blendVal = texture2D(blendMap, blendMapUV).a;
#if !@isFirstLayer
if (blendVal <= 0.002)
discard;
#endif
gl_FragData[0].a *= blendVal;
#endif
```
Because secondary passes use `EqualDepth` and depth write is disabled, discarding non-contributing fragments has zero effect on the depth buffer, immediately halts all subsequent texture sampling and lighting for that pixel, and requires zero changes to the underlying scene graph architecture.
### Performance Benefit Estimation & Empirical Test Results
- **Status:** **Tested & Rejected (+0.2% – +0.8% FPS — Ineffective)**
- **Empirical Test Results (Apple Silicon Hardware Timedemos):**
- Balmora: **57.66 FPS** vs **57.55 FPS** (+0.19% gain, within noise)
- Pelagiad (heavy terrain): **72.60 FPS** vs **72.00 FPS** (+0.83% gain)
- **Why It Failed:** Multi-pass terrain splatting still pays the CPU traversal and GPU vertex cost of issuing 2–4 draw calls per chunk. On TBDR GPUs, `discard` causes quad divergence along soft blend boundaries and disables tile fast-paths. True terrain performance requires Single-Pass Multi-Texturing (Opportunity #1).
- **Implementation Complexity:** Very Low
---
## 11. [FAILED / REJECTED] Front-to-Back Sorting for Opaque RenderBin (`RenderBin 0`)
> [!CAUTION]
> **Status: Tested & Rejected (-11.0% FPS Regression — Severe Performance Loss)**
> **Empirical Result:** 44.6 FPS vs 50.1 FPS baseline in Balmora timedemo.
> **Post-Mortem:** Sorting drawables by view distance broke OpenSceneGraph's `SORT_BY_STATE`, causing severe OpenGL shader, texture, and state thrashing on the CPU draw traversal and GPU driver. Furthermore, on Apple Silicon's Tile-Based Deferred Renderer (TBDR), the hardware rasterizer already performs on-chip Hidden Surface Removal (HSR) prior to fragment shading, rendering software front-to-back sorting completely redundant and strictly detrimental. **Do not re-attempt.**
### Overview & Problem
OpenMW renders opaque geometry in forward mode using [`MWRender::RenderBin_Default`](file:///Users/stellz/repos/openmw/apps/openmw/mwrender/renderbin.hpp#L11) (bin 0), which defaults to OSG's `SORT_BY_STATE`.
`SORT_BY_STATE` groups drawables by OpenGL StateSet/Program to minimize state switching. However, it results in arbitrary spatial rendering order.
In dense settlements (e.g., Balmora, Vivec, Ald'ruhn, Mournhold), large foreground statics (city walls, canton ramps, houses, temple steps) frequently occlude hundreds of smaller statics, building details, NPCs, and terrain behind them.
When occluded geometry is rasterized first, the GPU executes full forward fragment shaders (normal mapping, specular, ambient, 4-cascade shadows, fog) on millions of samples, only to overwrite them later when foreground walls are drawn.
On high-resolution displays (1440p, 4K, 3456×2160 Retina) and particularly with MSAA (2x, 4x, or 8x sample multiplication), this causes severe overdraw and memory bandwidth saturation.
### Proposed Solution
Configure the default opaque bin to use `osgUtil::RenderBin::SORT_FRONT_TO_BACK` (or install a custom prototype bin).
Sorting leaves front-to-back costs negligible CPU overhead (sorting depth values in the cull traversal) and allows GPU hardware Early-Z (Hierarchical-Z on AMD/NVIDIA; Tile-Based Deferred Hidden Surface Removal on Apple Silicon) to reject occluded geometry before fragment shader execution begins.
### Performance Benefit Estimation & Test Results
- **Status:** **Tested & Ineffective (-11.0% FPS Regression — Severe Performance Loss)**
- **Empirical Test Result:** Tested on hardware in Balmora (framerate dropped from 50.1 FPS to 44.6 FPS).
- **Why It Failed:** Sorting drawables by view distance destroyed OSG's `SORT_BY_STATE`, causing severe OpenGL shader, texture, and state thrashing. Furthermore, on Apple Silicon's Tile-Based Deferred Renderer (TBDR), the hardware already performs on-chip Hidden Surface Removal (HSR) prior to fragment shading, rendering software front-to-back sorting completely redundant and strictly detrimental.
---
## 12. [COMPLETED] Linear Depth Cascade Split Early-Out & Distance Culling Bugfix in `shadows_fragment.glsl`
> [!NOTE]
> **Status: Completed & Verified**
> **Empirical Result:** **-1.11 to -1.53 ms GPU time** in Pelagiad wilderness (-7.0% to -8.6% GPU speedup, 16.58 ms avg vs 17.69 ms baseline), **Peak 79.8 FPS** (+2.4 FPS). Performance neutral in Balmora (57.9 vs 58.1 FPS avg, 22.45 vs 22.50 ms GPU time).
> **Implemented Changes:**
> 1. In [`MWShadowTechnique::cull`](file:///Users/stellz/repos/openmw/components/sceneutil/mwshadowtechnique.cpp), recorded `cascadeFar * 1.05` for each cascade in `osg::Vec4 cascadeSplits` and bound as a single receiver uniform.
> 2. In [`shadows_fragment.glsl`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/shadows_fragment.glsl), guarded each cascade iteration with `if (!doneShadows && (cascadeSplits[@shadow_texture_unit_index] <= 0.0 || distance <= cascadeSplits[@shadow_texture_unit_index]))`. Far fragments skip perspective division and bounds checks for earlier cascades completely.
> 3. Fixed long-standing shadow distance culling defect: [`terrain.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/terrain.frag), [`groundcover.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/groundcover.frag), [`water.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/water.frag), and [`bs/default.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/bs/default.frag) were erroneously passing clip-space `linearDepth` (0.0 to 1.0) into `unshadowedLightRatio(float distance)` instead of `-passViewPos.z`. Replaced with `-passViewPos.z`, unlocking early distance fade/exit beyond 8192 units across all landscape, grass, trees, and water.
### Overview & Problem
In [`shadows_fragment.glsl`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/shadows_fragment.glsl#L26-L43), `unshadowedLightRatio(distance)` unrolls an iteration across all 4 shadow cascades:
```glsl
@foreach shadow_texture_unit_index @shadow_texture_unit_list
if (!doneShadows) {
vec3 shadowXYZ = shadowSpaceCoords@shadow_texture_unit_index.xyz / shadowSpaceCoords@shadow_texture_unit_index.w;
if (all(lessThan(shadowXYZ.xy, vec2(1.0, 1.0))) && all(greaterThan(shadowXYZ.xy, vec2(0.0, 0.0)))) {
shadowing = min(shadow2DProj(shadowTexture@shadow_texture_unit_index, shadowSpaceCoords@shadow_texture_unit_index).r, shadowing);
doneShadows = ...;
}
}
@endforeach
```
Cascades are partitioned along camera view-space depth `distance` (which is already passed into the function).
Yet every distant fragment (e.g., at 3500 units) computed perspective division and bounds testing for cascade 0, cascade 1, and cascade 2 before finally reaching cascade 3.
Furthermore, a severe distance culling defect existed in `terrain.frag`, `groundcover.frag`, `water.frag`, and `bs/default.frag`: they were passing `linearDepth` (which is NDC clip depth $\approx 0.99$) into `unshadowedLightRatio()`, meaning distant landscape, grass, and trees on the horizon never triggered the distance fade early-out and evaluated all 4 shadow cascades pointlessly!
### Implemented Solution
1. Computed `cascadeFar` distances in [`MWShadowTechnique::cull`](file:///Users/stellz/repos/openmw/components/sceneutil/mwshadowtechnique.cpp) and set a single `uniform vec4 cascadeSplits;` on the receiver StateSet.
2. In [`shadows_fragment.glsl`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/shadows_fragment.glsl), guarded cascade iterations so that fragments beyond a cascade's far split immediately skip light-space division and bounds checking:
```glsl
#if @shadow_texture_unit_index < 4
if (!doneShadows && (cascadeSplits[@shadow_texture_unit_index] <= 0.0 || distance <= cascadeSplits[@shadow_texture_unit_index]))
#else
if (!doneShadows)
#endif
```
3. Passed `-passViewPos.z` (eye distance in world units) to `unshadowedLightRatio` across all shaders, enabling immediate distance culling beyond maximum shadow distance.
### Performance Benefit & Empirical Hardware Results
- **Pelagiad (`--timedemo=foliage` — Open Wilderness with Distant Foliage/Terrain):**
- Average Framerate: **77.1 – 79.8 FPS** (Peak: **79.8 FPS**, +2.4 FPS over baseline)
- Frame Time: **12.53 – 13.25 ms** (down from 12.92 ms baseline)
- GPU Timer: **15.42 – 16.63 ms** (average **16.58 ms** vs **17.69 ms** baseline, **-1.11 to -1.53 ms GPU savings**)
- Draw Traversal: **10.60 – 10.90 ms**
- **Balmora (`--timedemo=balmora` — Dense Urban Architecture):**
- Average Framerate: **57.9 – 60.0 FPS** (average **57.9 FPS** vs 58.1 FPS baseline)
- GPU Timer: **21.86 – 23.14 ms** (average **22.45 ms** vs 22.50 ms baseline)
- Draw Traversal: **15.43 – 16.74 ms** (average **16.06 ms** vs 16.00 ms baseline)
- **Visual Verification:** Pixel-perfect with zero edge artifacts or cascade seams in both [`screenshot034.png`](file:///Users/stellz/Library/Application Support/openmw/screenshots/screenshot034.png) and [`screenshot035.png`](file:///Users/stellz/Library/Application Support/openmw/screenshots/screenshot035.png).
---
## 13. [COMPLETED] Per-Frame Invariant Hoisting and Water Depth Pruning in `fog.glsl`
> [!NOTE]
> **Status: Completed & Verified (Commit [`6d8a54cce2`](file:///Users/stellz/repos/openmw))**
> **Empirical Result:** **+8.4% FPS** (58.50 FPS vs 53.97 FPS baseline), **-1.59 ms GPU time** (-6.4%), **-1.45 ms Draw Traversal**.
> **Implemented Changes:** Hoisted Z-row dot product to eliminate 4x4 matrix multiply per fragment across all surfaces in water cells; precomputed depth correction constants into compile-time constants; guarded radial distance evaluation.
### Overview & Problem
In [`files/shaders/compatibility/fog.glsl`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/fog.glsl#L42-L78), `computeFog()` is called on every fragment across [`objects.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/objects.frag#L245), [`terrain.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/terrain.frag#L107), and [`groundcover.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/groundcover.frag#L88).
Inside `computeFog()`, several values that are invariant per frame or per draw call are redundantly computed per pixel:
1. `cameraPos = osg_ViewMatrixInverse[3].xyz; cameraBelowWater = cameraPos.z < waterHeight;` is re-evaluated inside every fragment.
2. `worldPos = (osg_ViewMatrixInverse * vec4(pos, 1)).xyz;` performs a full 4x4 matrix-vector multiplication merely to check `worldPos.z < waterHeight`. Only the Z row is needed (`dot(osg_ViewMatrixInverse[2].xyz, pos) + osg_ViewMatrixInverse[3].z`).
3. `1.0 / (end - start)` is computed per fragment in linear fog instead of uploading `fogInvRange` once from the CPU.
### Proposed Solution
1. Pass `cameraBelowWater` as a uniform boolean rather than recalculating it from `osg_ViewMatrixInverse[3].xyz`.
2. Compute only the Z component of world position when testing water height: `float worldZ = dot(osg_ViewMatrixInverse[2].xyz, pos) + osg_ViewMatrixInverse[3].z`.
3. Precalculate `fogInvRange = 1.0 / (end - start)` on the CPU.
### Performance Benefit Estimation & Empirical Results
- **Status:** **Completed & Verified (`6d8a54cce2`)**
- **Empirical Test Result:** **+8.4% FPS** (58.50 FPS vs 53.97 FPS baseline), **-1.59 ms GPU frame time** (-6.4%), **-1.45 ms Draw Traversal**.
- **Test Scenario:** Balmora exterior timedemo on hardware.
- **Bottlenecks Relieved:** Fragment ALU pressure, register pressure, redundant 4x4 matrix-vector multiplication in scenes with water.
---
## 14. [FAILED / REJECTED] Universal Directional Light & Shader Math Invariant Hoisting
> [!CAUTION]
> **Status: Tested & Rejected (0.0% FPS gain — Pure Noise Floor)**
> **Empirical Result:**
> - Pelagiad: 76.6 FPS avg vs 76.5 FPS clean baseline (13.05 ms vs 13.06 ms frame time; GPU timer: 17.02 ms vs 17.86 ms).
> - Balmora: 56.8 FPS avg vs 56.2 FPS clean baseline (17.59 ms vs 17.80 ms frame time; GPU timer: 23.65 ms vs 23.62 ms).
> - Draw Traversal was completely unaffected (~11.0 ms in Pelagiad, ~17.0 ms in Balmora).
> **Post-Mortem:** Pre-normalizing the sun vector on CPU once per frame, guarding dead screen-space division, and hoisting loop-invariant shadow offsets saved a small number of scalar ALU instructions. On modern desktop and TBDR GPUs, simple scalar arithmetic is completely hidden behind texture sampling, cache misses, and memory latency. All measured delta was within run-to-run system and thermal variance (< 0.1 ms). **Reverted to avoid codebase churn.**
### Overview & Problem
Fragment and vertex shaders across OpenMW repeatedly compute mathematical operations on uniform variables and view-space coordinates that are completely invariant per frame or per draw call:
1. **Per-Fragment Directional Light Normalization**: In [`calcDirectionalLighting`](file:///Users/stellz/repos/openmw/files/shaders/lib/light/util.glsl#L64):
```glsl
void calcDirectionalLighting(DirectionalLight light, ...) {
vec3 dir = normalize(light.position.xyz);
// ...
}
```
`light.position` is `sun.position`, a uniform updated once per frame on the CPU in [`LightManager::update`](file:///Users/stellz/repos/openmw/components/sceneutil/lightmanager.cpp#L287) via `sun->getPosition() * (*viewMatrix)`. Every single shaded fragment across all objects, terrain, water, and groundcover executes `rsqrt` and 3 multiplications just to normalize this static uniform.
2. **Duplicate View Vector Normalization in `objects.frag`**:
At line 177, `objects.frag` executes `vec3 viewVec = normalize(passViewPos);`. Later at line 234, it calls `doLighting(..., passViewPos, ...)`, which inside [`lighting_fragment_impl.glsl`](file:///Users/stellz/repos/openmw/files/shaders/lib/core/lighting_fragment_impl.glsl#L18) calculates `vec3 viewDir = normalize(viewPos);` a second time for the exact same fragment.
3. **Dead Math in `objects.frag`**:
Line 136 computes `vec2 screenCoords = gl_FragCoord.xy / screenRes;` unconditionally for every fragment, but `screenCoords` is only referenced inside `#if defined(DISTORTION) && DISTORTION` and `#if !defined(FORCE_OPAQUE) && @softParticles`. On 99% of all scene geometry, this screen-space division is dead overhead.
4. **Loop-Invariant Vertex Math in `shadows_vertex.glsl`**:
In [`shadows_vertex.glsl`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/shadows_vertex.glsl#L30-L42), `shadowOffset = vec4(viewNormal * @shadowNormalOffset, 0.0);` and `vec4 offsetViewPosition = viewPos + shadowOffset;` are evaluated inside the 4-iteration cascade loop. Both calculations are completely independent of the cascade index.
### Proposed Solution
1. In `LightManager::update`, pre-normalize the view-space sun direction vector on the CPU before setting `sun.position`:
```cpp
osg::Vec4f sunViewPos = sun->getPosition() * (*viewMatrix);
osg::Vec3f sunDir(sunViewPos.x(), sunViewPos.y(), sunViewPos.z());
sunDir.normalize();
stateset->getUniform("sun.position")->set(osg::Vec4f(sunDir, sunViewPos.w()));
```
In `calcDirectionalLighting`, read `vec3 dir = light.position.xyz;` directly without `normalize()`.
2. In `objects.frag`, guard `screenCoords` with `#if (defined(DISTORTION) && DISTORTION) || (!defined(FORCE_OPAQUE) && @softParticles)`.
3. In `shadows_vertex.glsl`, hoist `shadowOffset` and `offsetViewPosition` outside the `@foreach shadow_texture_unit_index` loop.
---
## 15. [COMPLETED / RESOLVED] Active-Grid Static Architecture & Clutter Batching in Dense Settlements (Focus 2: Draw Call Elimination for Balmora)
### Overview & Problem
In dense settlements like Balmora, performance is severely bottlenecked on the CPU by OSG Draw Traversal (**~16.0 ms frame time**, **~1,200 draw calls** across main and shadow passes).
A huge percentage of these draw calls are static architectural components (Hlaalu building walls, roof segments, stone stairs, canal bridge arches, stone embankments, lampposts, and urns) placed in the active 3x3 exterior cell grid.
Currently, [`ObjectPaging::createChunk`](file:///Users/stellz/repos/openmw/apps/openmw/mwrender/objectpaging.cpp#L790-L793) evaluates whether to merge static instances using:
```cpp
const float mergeCost = analyzeResult.mNumVerts * size;
const float mergeBenefit = analyzeVisitor.getMergeBenefit(analyzeResult) * mMergeFactor;
const bool merge = mergeBenefit > mergeCost;
```
Because `mMergeFactor` defaults to `250`, any static mesh with more than a few hundred vertices (e.g., a 1,200-vertex stone canal wall or Hlaalu stair segment with 2 instances) fails the `mergeBenefit > mergeCost` check. When `merge` is false, [`objectpaging.cpp`](file:///Users/stellz/repos/openmw/apps/openmw/mwrender/objectpaging.cpp#L862) attaches each instance as an individual `osg::MatrixTransform` to `group` (bypassing `SceneUtil::Optimizer::MERGE_GEOMETRY`).
Furthermore, in active grid cells (size = 1 cell), the player is located right inside the town. Fine-grained per-building frustum culling provides negligible culling benefits because nearly all buildings in the active cell are within the camera frustum, yet OpenMW pays the cost of traversing and submitting 800+ separate draw calls every frame.
### Implemented Solution & Results
1. **Engine Default Parameter Tuning (`object paging merge factor = 500`)**:
- Comprehensive multi-trial sweep conducted across merge factors 250, 400, 500, 640, 1000, and 2000 (documented in [`docs/object_paging_merge_factor_sweep.md`](file:///Users/stellz/repos/openmw/docs/object_paging_merge_factor_sweep.md)).
- Merge factor `500` proved to be the optimal global sweet spot, allowing duplicate modular static architecture to merge without overflowing bounding boxes.
- **Committed in [`4d8ecf6435`](file:///Users/stellz/repos/openmw)**.
2. **Automated Timedemo Draw Call Tracking**:
- Instrumented [`Timedemo`](file:///Users/stellz/repos/openmw/apps/openmw/timedemo.cpp) to extract per-frame visible drawable counts from OSG `Renderer` statistics.
- **Committed in [`21d698ea55`](file:///Users/stellz/repos/openmw)**.
3. **Active-Grid Optimizer Batching (Option B — Tested & Reverted)**:
- Evaluated 4-quadrant spatial sub-cell clustering and forced active-grid batching. In active exterior cells, `SceneUtil::Optimizer` interacts with active `RefnumMarker` user data and transform hierarchies, preventing batch flattening and inflating draw calls (4,857 calls). Reverted cleanly in favor of the clean factor 500 baseline.
### Performance Benefit & Hardware Verification
- **Status:** **Completed & Verified (Commits [`4d8ecf6435`](file:///Users/stellz/repos/openmw) and [`21d698ea55`](file:///Users/stellz/repos/openmw))**
- **Balmora (`--timedemo=balmora`):**
- Draw Calls: **2,983 calls/frame** vs **3,019 baseline** (**-36 draw calls/frame eliminated** across passes)
- Framerate: **57.5 – 58.9 FPS avg** (peak **60.0 FPS**) vs **56.5 FPS baseline**
- Draw Traversal: **15.83 – 16.47 ms** vs **16.63 ms baseline** (peak down to 15.50 ms)
- GPU Timer: **22.33 – 23.06 ms** vs **23.27 ms baseline** (peak down to 22.02 ms)
- **Pelagiad (`--timedemo=foliage`):**
- Draw Calls: **2,105 – 2,109 calls/frame**
- Framerate: **77.0 – 78.1 FPS** vs **76.7 FPS baseline**
- Draw Traversal: **10.33 – 10.84 ms**
- GPU Timer: **16.25 – 17.15 ms**
- **Visual Status:** Completely glitch-free (verified via [`screenshot040.png`](file:///Users/stellz/Library/Application Support/openmw/screenshots/screenshot040.png) and [`screenshot041.png`](file:///Users/stellz/Library/Application Support/openmw/screenshots/screenshot041.png)).
---
## 16. Shared Shadow Receiver StateSet & StateGraph Pruning (Focus 2: Draw Traversal Optimization)
### Overview & Problem
When cascaded shadows are enabled, OpenMW attaches shadow receiving state via [`MWShadowTechnique::prepareStateSetForRenderingShadow`](file:///Users/stellz/repos/openmw/components/sceneutil/mwshadowtechnique.cpp#L3247-L3273).
Every frame, this StateSet is populated with 15+ individual uniforms (`shadowTexture0..3`, `shadowSpaceMatrix0..3`, `validRegionMatrix0..3`, `cascadeSplits`, `maximumShadowMapDistance`, `shadowFadeStart`).
Previously, this StateSet was pushed directly onto the CullVisitor stack via `cv.pushStateSet(shadowReceiverStateSet)` during `cullShadowReceivingScene`.
During cull traversal, as thousands of scene graph nodes were visited, OSG's `StateGraph` constructed an intermediate state level for every drawable inheriting shadow uniforms. During draw traversal in Balmora (with 1,200+ drawables), the rendering thread had to evaluate and navigate through this intermediate StateGraph layer on every StateGraph transition.
### Implemented Solution
1. Hoisted the shadow receiver StateSet to the top of the CullVisitor active `RenderStage` via `renderStage->setStateSet(shadowReceiverStateSet)` in both `MWShadowTechnique::cull` and `copyShadowMap`.
2. Removed `cv.pushStateSet(shadowReceiverStateSet)` / `cv.popStateSet()` from `cullShadowReceivingScene`, pruning an entire intermediate StateGraph level across all drawables in the scene.
3. Added `osg::StateAttribute::ON | osg::StateAttribute::OVERRIDE` to shadow uniforms and shadow map texture units in `prepareStateSetForRenderingShadow`.
### Performance Benefit Estimation & Empirical Results
- **Status:** **Completed & Verified (Commit [`7bb3541b65`](file:///Users/stellz/repos/openmw))**
- **Subsystem / Pass Reduction:** Eliminates intermediate StateGraph level across 1,200+ drawables.
- **Empirical Hardware Test Results (Apple Silicon, Multi-Trial Averages):**
- **Pelagiad (`--timedemo=foliage`):**
- Draw Traversal: **10.63 ms** vs **11.09 ms** baseline (**-0.46 ms Draw reduction**, best run 10.17 ms)
- GPU Timer: **16.84 ms** vs **17.86 ms** baseline (**-1.02 ms GPU reduction**)
- Framerate: **76.7 FPS avg** (peak 77.7 FPS)
- **Balmora (`--timedemo=balmora`):**
- Draw Traversal: **16.63 ms** vs **17.02 ms** baseline (**-0.39 ms Draw reduction**, best run 15.84 ms)
- GPU Timer: **23.27 ms** vs **23.62 ms** baseline (**-0.35 ms GPU reduction**)
- Framerate: **56.5 FPS avg** (peak 58.6 FPS)
- **Bottlenecks Relieved:** CPU Draw Traversal StateGraph hierarchy navigation overhead, redundant uniform state evaluations.
---
## 17. Architectural Finding: Main Camera vs. Shadow Pass Profile & The `StateGraph` Bottleneck
> [!NOTE]
> **Empirical Discovery (Automated OSG & Timedemo Profiling on Apple Silicon):**
> Disabling cascaded shadows completely in Balmora eliminates **1,338 draw calls/frame (-44.9%)** and **528,487 triangles (-56.3%)**, yet average framerate is **completely unchanged (58.5 FPS shadows on vs 57.7 FPS shadows off)**.
> The CPU render thread is entirely pinned by the **1,645 Main Camera draw calls** and their **509 unique `StateGraph` state transitions**.
### Empirical Profile: Shadows Enabled vs. Disabled (Balmora Hardware Timedemo)
| Metric | Shadows Enabled (`enable shadows = true`) | Shadows Disabled (`enable shadows = false`) | Delta |
| :--- | :--- | :--- | :--- |
| **Total Visible Drawables** | **2,983** | **1,645** | **-1,338 (-44.9%)** |
| **Visible Triangles** | 938,767 | 410,280 | **-528,487 (-56.3%)** |
| **Visible Vertex Count** | 836,324 | 356,383 | **-479,941 (-57.4%)** |
| **OSG StateGraphs (State Changes)** | **536** | **509** | **-27 (-5.0%)** |
| **CPU Cull Traversal** | 2.02 ms | 1.46 ms | -0.56 ms (-27.6%) |
| **CPU Draw Traversal** | **16.04 – 16.74 ms** | **16.35 – 17.77 ms** | **Virtually Unchanged (~16–17 ms)** |
| **GPU Draw Time** | 22.44 ms | 21.88 ms | -0.56 ms (-2.5%) |
| **Average Framerate** | **58.5 FPS** (17.09 ms) | **57.7 FPS** (17.32 ms) | **Identical (~58 FPS)** |
### Architectural Root Cause Analysis
1. **State Switches Dominate CPU Cost, Not Raw Draw Count (The `StateGraph` Bottleneck):**
- In [`osgUtil::RenderBin`](file:///Users/stellz/repos/openmw/components/sceneutil/mwshadowtechnique.cpp#L180-L211), drawables are sorted by [`StateGraph`](file:///Users/stellz/repos/openmw/components/sceneutil/mwshadowtechnique.cpp#L213-L230). Each `StateGraph` node represents a distinct OpenGL state (shader program, texture unit bindings, material properties, blend modes).
- In Balmora, the **1,645 main camera drawables** are fragmented across **509 unique StateGraphs** (~1 heavy state transition every 3 draw calls). On macOS OpenGL over Metal, state transitions (`glUseProgram`, `glBindTexture`, uniform updates) incur heavy CPU overhead on the render thread.
- In contrast, the **1,338 shadow drawables** share almost identical depth-only state sets across all 4 cascades, contributing only **27 StateGraphs** in total.
- The CPU Draw thread spends **~16.3 ms of its 16.8 ms solely dispatching the Main Camera's 509 state changes**. The shadow passes execute in <0.5 ms of CPU time.
2. **Depth-Only Passes vs. Full Forward Shading (GPU Cost Asymmetry):**
- **Shadow Cascades:** Render depth-only into an offscreen FBO. Shaders are empty (no lighting, specular, normal maps, fog, or splatting). Modern GPUs execute depth-only rasterization at double or quadruple rate (Hi-Z fast path). All 528k shadow triangles cost only **~0.70 ms GPU time**.
- **Main Camera Pass:** Shading 410k fragments involves forward lighting, point lights, sun specular, 4-cascade shadow map sampling with PCF filtering, terrain splatting, and fog. The main camera accounts for **>95% of total GPU rasterization cost**.
3. **Strategic Optimization Implication:**
- Culling shadow casters (beyond existing small-feature culling) does not move the needle on framerate in dense exterior towns like Balmora.
- To achieve measurable FPS gains in CPU-bottlenecked scenes, optimizations must target the **1,645 Main Camera draw calls and their 509 StateGraphs** (e.g. reducing redundant StateSets, sharing materials, or batching exterior static clutter/architecture so that geometry and states merge).
---
## 18. [COMPLETED] Material Uniform Deduplication & Shared Uniform Caching (Focus: Main Camera StateGraph & CPU Draw Reduction)
### Overview & Problem
In the CPU-bottlenecked Balmora exterior, profiling revealed that the Main Camera pass alone dispatches **506 StateGraphs** across **235 unique StateSets**.
A major contributor to this state fragmentation was the `SceneUtil::Material` uniform subsystem:
- Over **311 StateSets** in the scene contain individual instances of 8 material uniforms (`material.diffuse`, `material.ambient`, `material.specular`, `material.emission`, `material.shininess`, `material.emissiveMult`, `material.specStrength`, `material.vertexColorMode`).
- In Morrowind NIF assets, nearly all static architecture, rocks, bridges, and clutter share standard material properties (diffuse = `(1,1,1,1)`, ambient = `(1,1,1,1)`, specular = `(0,0,0,0)`, emission = `(0,0,0,1)`). However, because NIF models have vertex lighting, `mat->setVertexColorMode(AmbientAndDiffuse)` is set in [`nifloader.cpp`](file:///Users/stellz/repos/openmw/components/nifosg/nifloader.cpp#L2738), causing `*mat != defaultMat` to evaluate to true.
- When [`ShaderVisitor::createProgram`](file:///Users/stellz/repos/openmw/components/shader/shadervisitor.cpp#L445-L447) encountered an `osg::StateAttribute::MATERIAL`, it invoked [`Material::setStateSet`](file:///Users/stellz/repos/openmw/components/sceneutil/material.cpp#L22-L38), which called `stateset->getOrCreateUniform(...)` for all 8 uniforms.
- In OpenSceneGraph, `StateSet::compare` compares uniform sets by sorting and comparing pointer addresses (`RefUniformPair.first`). Because `getOrCreateUniform` allocated 8 fresh `osg::Uniform` heap objects for every StateSet, two StateSets with identical material properties failed pointer comparison, forcing OSG to allocate separate StateGraphs and issuing hundreds of redundant uniform updates during Draw Traversal.
### Implemented Solution
1. In [`components/sceneutil/material.hpp`](file:///Users/stellz/repos/openmw/components/sceneutil/material.hpp) and [`components/sceneutil/material.cpp`](file:///Users/stellz/repos/openmw/components/sceneutil/material.cpp), added 8 cached `osg::ref_ptr` members (`mUniformDiffuse`, `mUniformAmbient`, `mUniformSpecular`, `mUniformEmission`, `mUniformShininess`, `mUniformEmissiveMult`, `mUniformSpecStrength`, `mUniformVertexColorMode`) to the `SceneUtil::Material` attribute.
2. Implemented lazy, thread-safe creation via `Material::ensureUniforms()` protected by `std::mutex mUniformMutex`.
3. Updated `Material::setStateSet` and `Material::updateStateSet` to attach the cached `osg::Uniform` pointers (`stateset->addUniform(mUniform...)`) instead of creating fresh unshared uniform heap objects.
4. Updated all out-of-line setters (`setDiffuse`, `setAmbient`, `setSpecular`, `setEmission`, `setShininess`, `setEmissiveMult`, `setSpecStrength`, `setVertexColorMode`) to dynamically synchronize the cached uniforms under the mutex when present.
5. This allows OSG's `StateSet::compare` to recognize uniform equality via pointer matching, eliminating thousands of redundant heap allocations per scene load and accelerating Draw Traversal uniform dispatch.
### Performance Benefit Estimation & Empirical Results
- **Status:** **Completed & Verified (Commit [`d1bd976ec6`](file:///Users/stellz/repos/openmw))**
- **Subsystem Impact:** Eliminates thousands of heap allocations per exterior cell; allows OSG `StateSet::compare` pointer equality matching; relieves CPU Draw Traversal overhead.
- **Empirical Hardware Test Results (Apple Silicon, Multi-Trial Averages):**
- **Balmora (`--timedemo=balmora` — 60 frames, 30 warmup):**
- Framerate: **58.7 FPS avg** (peak **60.4 FPS** / 16.55 ms) vs 58.3 FPS baseline
- Draw Traversal: **15.35 ms** vs **15.80 ms** baseline (**-0.45 ms Draw reduction**)
- GPU Timer: **21.72 ms** vs **22.20 ms** baseline (**-0.48 ms GPU reduction**)
- Draw Calls: **2,979 – 2,983 calls/frame** (-4 calls eliminated)
- **Pelagiad (`--timedemo=foliage` — 60 frames, 30 warmup):**
- Framerate: **78.4 FPS avg** (12.74 ms avg frame time)
- Draw Traversal: **10.88 ms**
- GPU Timer: **17.59 ms**
- **Visual Fidelity:** Flawless rendering verified via timedemo screenshot inspection; material diffuse, ambient, specular, emissive, and vertex color modes are identical.
---
## 19. [REJECTED] StateSet Flattening & Hierarchy Consolidation for Static Scene Graphs
> [!WARNING]
> **Status: Rejected / Abandoned After Empirical Hardware Testing**
> **Empirical Result:** **+196 draw calls in Balmora (+6.6%, 3,179 vs 2,983)** and **+128 draw calls in Pelagiad (+6.1%, 2,237 vs 2,109)**. Draw traversal improved modestly (-0.25 ms in Balmora, 15.27 vs 15.52 ms), and GPU time decreased slightly (-0.13 ms, 21.80 vs 21.93 ms), but framerate was unchanged (60.5 vs 60.4 FPS in Balmora, 77.7 vs 77.4 FPS in Pelagiad) due to severe batch fragmentation.
> **Root Cause:** In stock OpenMW, leaf geometries under static models share `nullptr` StateSets. During exterior chunk creation in [`objectpaging.cpp`](file:///Users/stellz/repos/openmw/apps/openmw/mwrender/objectpaging.cpp), [`MergeGeometryVisitor`](file:///Users/stellz/repos/openmw/components/sceneutil/optimizer.cpp#L1444) uses [`LessGeometry`](file:///Users/stellz/repos/openmw/components/sceneutil/optimizer.cpp#L1143) to sort and merge compatible static geometries strictly by **raw pointer equality** (`lhs->getStateSet() < rhs->getStateSet()`). All `nullptr` geometries match and merge into large unified batches. Pushing StateSets down to leaf geometries in [`SceneManager::getTemplate`](file:///Users/stellz/repos/openmw/components/resource/scenemanager.cpp) resulted in distinct heap-allocated StateSet pointers across different template models. `LessGeometry` saw mismatched pointers between different objects and refused to batch them, fracturing chunk batches into hundreds of unmerged draw calls. Fixing this would require an engine-wide StateSet value interning system across all paged cells—an enormous complexity and regression risk for negligible gain.
### Overview & Problem
In OpenMW's exterior scene graph, nodes are arranged hierarchically (`Cell -> Group -> MatrixTransform -> Group -> Geometry`).
During cull traversal, OSG's `CullVisitor` pushes StateSets as it descends the tree (`_currentStateGraph = _currentStateGraph->find_or_insert(ss)`).
Profiling in Balmora showed an expansion ratio: **235 unique StateSets expand into 506 StateGraphs**.
This occurs because intermediate static parent nodes (transforms, groups) possess their own StateSets (e.g. for textures, blending, or shader hints), while leaf geometries hold their own material and texture StateSets. Even when two child drawables share the exact same geometry StateSet, if their parent node paths differ, OSG branches into separate `StateGraph` subtrees.
During Draw Traversal in [`osgUtil::RenderBin::draw`](file:///Users/stellz/repos/openmw/components/sceneutil/mwshadowtechnique.cpp#L180-L211), the render thread must recursively ascend and descend these intermediate StateGraph levels, applying and popping states on the CPU.
### Empirical Post-Mortem & Architecture Findings
An implementation of `FlattenStateSetsVisitor` was developed in [`components/sceneutil/optimizer.cpp`](file:///Users/stellz/repos/openmw/components/sceneutil/optimizer.cpp) and integrated into [`components/resource/scenemanager.cpp`](file:///Users/stellz/repos/openmw/components/resource/scenemanager.cpp).
While it successfully eliminated intermediate node StateSets and pushed them down to leaf drawables, the interaction with exterior cell batching produced severe unintended consequences:
1. **Pointer Incompatibility in Geometry Merging:**
In [`components/sceneutil/optimizer.cpp`](file:///Users/stellz/repos/openmw/components/sceneutil/optimizer.cpp#L1145):
```cpp
struct LessGeometry {
bool operator() (const osg::ref_ptr& lhs, const osg::ref_ptr& rhs) const {
if (lhs->getStateSet() < rhs->getStateSet()) return true;
if (rhs->getStateSet() < lhs->getStateSet()) return false;
// ...
```
Stock OpenMW's `MergeGeometryVisitor` groups drawables by pointer. Because geometries from different models (e.g., `wall_corner_01.nif` vs `wall_straight_01.nif`) received distinct StateSet instances with distinct memory addresses, `LessGeometry` segregated them into separate buckets, preventing them from being merged across the chunk.
2. **Draw Call Explosion:**
- Balmora: **+196 draw calls (+6.6%)** (3,179 vs 2,983 baseline)
- Pelagiad: **+128 draw calls (+6.1%)** (2,237 vs 2,109 baseline)
Disabling `FLATTEN_STATESETS` via `OPENMW_OPTIMIZE="~FLATTEN_STATESETS"` immediately restored the exact 2,983 baseline, proving conclusively that StateSet flattening was the direct cause.
3. **Conclusion:** StateSet flattening is counter-productive in OpenSceneGraph when combined with raw-pointer-based geometry merging. The optimization has been abandoned.
---
## 20. [FAILED / REJECTED] Rasterizer Interpolator Pruning (`normalToViewMatrix` Elimination)
### Overview & Problem
In [`files/shaders/compatibility/normals.glsl`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/normals.glsl#L1), the varying:
```glsl
varying mat3 normalToViewMatrix;
```
is declared unconditionally and included in [`objects.vert`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/objects.vert#L76), [`terrain.vert`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/terrain.vert#L27), [`groundcover.vert`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/groundcover.vert#L41), and [`bs/default.vert`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/bs/default.vert#L37).
In the vertex shader:
```glsl
normalToViewMatrix = gl_NormalMatrix;
```
A `mat3` varying occupies **3 full vec4 hardware interpolator registers (9 scalar floats)**. These 9 floats are interpolated across the rasterizer on every fragment of every mesh and terrain chunk in the game.
However, in [`objects.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/objects.frag#L133-L172) and [`terrain.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/terrain.frag#L154-L175), `normalToViewMatrix` is **only ever read when normal mapping or parallax mapping is compiled in**:
```glsl
#if @parallax || @diffuseParallax
offset = getParallaxOffset(transpose(normalToViewMatrix) * normalize(-passViewPos), height);
#endif
// ...
#if @normalMap
vec3 viewNormal = normalToView(normal);
#else
vec3 viewNormal = normalize(gl_NormalMatrix * passNormal);
#endif
```
For standard Morrowind assets (the vast majority of static architecture, landscape, and clutter, which have no normal/parallax maps), `normalToViewMatrix` is **completely unused in the fragment shader**. The GPU wastes vertex-to-fragment register bandwidth interpolating an entire 3x3 matrix on every pixel for nothing.
### Proposed Solution
1. Wrap `varying mat3 normalToViewMatrix;` in [`files/shaders/compatibility/normals.glsl`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/normals.glsl) with `#if NEED_NORMAL_TO_VIEW_MATRIX`.
2. Guard the assignment in [`objects.vert`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/objects.vert), [`terrain.vert`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/terrain.vert), [`groundcover.vert`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/groundcover.vert), and [`bs/default.vert`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/bs/default.vert) so it only executes when normal/parallax mapping is enabled.
3. In [`groundcover.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/groundcover.frag), compute `normalize(gl_NormalMatrix * passNormal)` when normal mapping is disabled.
### Empirical Validation & Test Results (8 Trials)
- **Status:** **Tested & Abandoned (Ineffective / Regressive)**
- **Baseline (Commit `baa9fe40d8`):**
- Pelagiad (`foliage`): Mean **76.42 FPS** (±1.39), Frame Time **13.09 ms**, GPU Timer **17.15 ms**
- Balmora (`balmora`): Mean **56.94 FPS** (±1.35), Frame Time **17.58 ms**, GPU Timer **23.13 ms**
- **Opportunity #20 (8-Run Benchmark):**
- Pelagiad (`foliage`): Mean **74.42 FPS** (±1.74) (**-2.00 FPS / -2.6%**), Frame Time **13.44 ms** (+0.35 ms), GPU Timer **16.73 ms**
- Balmora (`balmora`): Mean **54.55 FPS** (±0.99) (**-2.39 FPS / -4.2%**), Frame Time **18.33 ms** (+0.75 ms), GPU Timer **23.76 ms**
### Root Cause Analysis
1. **Driver Cross-Stage Dead Code Elimination (DCE):**
In modern GPU compilers (specifically Apple Silicon Metal underlying macOS OpenGL), varyings that are output by the vertex shader but never read in the fragment shader are pruned at pipeline link time. In `objects.frag`, `terrain.frag`, and `bs/default.frag`, `normalToViewMatrix` was already completely unread when `@normalMap` and `@parallax` were 0. The hardware rasterizer was therefore already stripping the unreferenced interpolators automatically.
2. **Dynamic Uniform Inlining in `groundcover.frag`:**
In `groundcover.frag`, where `normalToView(normalize(passNormal))` was previously invoked, replacing it with `normalize(gl_NormalMatrix * passNormal)` forced per-fragment dynamic uniform loading and a 3x3 matrix-vector multiplication in the fragment ALU for millions of grass fragments. Rather than saving time, this added fragment ALU overhead.
3. **Conclusion:**
Manual interpolator pruning provides zero bandwidth benefit due to hardware driver DCE, while adding fragment-stage ALU overhead. The optimization has been completely reverted to maintain clean baseline performance.
---
## 21. Dead Math & Redundant Normalization Elimination in `objects.frag`
### Overview & Problem
[`objects.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/objects.frag) executes for every static and dynamic object in the world. Several expensive vector calculations are evaluated unconditionally even when the corresponding visual features are disabled:
1. **Unconditional Screen-Space Coordinates:**
```glsl
vec2 screenCoords = gl_FragCoord.xy / screenRes;
```
`screenCoords` is computed on line 136 for every fragment, but is only read by `DISTORTION` (line 143) and `softParticles` (line 260). For 99% of static and opaque objects, this division is completely dead.
2. **Unconditional 3D Normalization:**
```glsl
vec3 viewVec = normalize(passViewPos);
```
Computed on line 177 via an expensive `rsqrt` and 3 scalar multiplies, but only read by `@envMap` (line 195) and `softParticles` (line 255).
3. **Redundant Normalization in Per-Pixel Lighting:**
Inside [`doLighting(...)`](file:///Users/stellz/repos/openmw/files/shaders/lib/core/lighting_fragment_impl.glsl#L18), `vec3 viewDir = normalize(viewPos);` is evaluated a second time even if `viewVec` was already normalized.
4. **Dead `viewNormal` Transformation in Vertex-Lit Shading:**
When `!PER_PIXEL_LIGHTING` is active (standard Morrowind objects) and `@envMap`, `@softParticles`, and Normal RT output (`!defined(FORCE_OPAQUE) && !@disableNormals`) are disabled, `vec3 viewNormal = normalize(gl_NormalMatrix * passNormal);` is evaluated on line 174 but never consumed.
### Proposed Solution
1. Guard `screenCoords` with `#if (defined(DISTORTION) && DISTORTION) || (!defined(FORCE_OPAQUE) && @softParticles)`.
2. Guard `viewVec` with `#if @envMap || (!defined(FORCE_OPAQUE) && @softParticles)`.
3. Guard `viewNormal` so that it is only computed when per-pixel lighting, env mapping, soft particles, or normal render targets are active.
### Performance Benefit Estimation
- **Expected Impact:** **-1% – 2% GPU fragment time** (1% – 2% fragment shading speedup), lower scalar ALU instruction count on every fragment.
- **Implementation Complexity:** **Very Low**.
---
## 22. Vertex Shadow Offset Hoisting in `shadows_vertex.glsl`
### Overview & Problem
In [`files/shaders/compatibility/shadows_vertex.glsl`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/shadows_vertex.glsl#L30):
```glsl
void setupShadowCoords(vec4 viewPos, vec3 viewNormal)
{
#if SHADOWS
vec4 shadowOffset;
@foreach shadow_texture_unit_index @shadow_texture_unit_list
// ...
shadowOffset = vec4(viewNormal * @shadowNormalOffset, 0.0);
vec4 offsetViewPosition = viewPos + shadowOffset;
shadowSpaceCoords@shadow_texture_unit_index = shadowSpaceMatrix@shadow_texture_unit_index * offsetViewPosition;
@endforeach
#endif
}
```
When shadows are active with 4 cascades, `shadowOffset` and `offsetViewPosition` are computed **4 times per vertex** inside the unrolled cascade loop.
### Proposed Solution
Hoist `shadowOffset` and `offsetViewPosition` once before the loop:
```glsl
#if !@disableNormalOffsetShadows
vec4 offsetViewPosition = viewPos + vec4(viewNormal * @shadowNormalOffset, 0.0);
#endif
```
### Performance Benefit Estimation
- **Expected Impact:** **<1% Vertex ALU**, minor cleanliness improvement.
- **Implementation Complexity:** **Very Low**.
---
## 23. [COMPLETED] Zero-Weight Diffuse Fetch Elimination in Single-Pass Terrain (`terrain.frag`)
> [!NOTE]
> **Status: Completed & Verified**
> **Empirical Result (8-Trial Timedemo Benchmark vs. Control Baseline `d1bd976ec6`):**
> - **Pelagiad (`--timedemo=foliage` — >60% Terrain):** **+1.71 FPS mean gain (+2.29%)** (76.42 FPS ±1.39 vs 74.71 FPS ±2.32 baseline), **-0.30 ms mean frame time** (13.09 ms vs 13.39 ms), peak **78.10 FPS**. Frametime standard deviation cut by **40%** (±1.39 vs ±2.32).
> - **Balmora (`--timedemo=balmora` — Heavy Architecture):** **+0.80 FPS mean gain (+1.42%)** (56.94 FPS ±1.35 vs 56.14 FPS ±1.02 baseline), **-0.25 ms mean frame time** (17.56 ms vs 17.81 ms), **-0.27 ms GPU time** (23.13 ms vs 23.40 ms), peak **59.80 FPS**.
> - **Visual Verification:** 100% pixel-perfect identical to baseline (verified via [`screenshot043.png`](file:///Users/stellz/Library/Application Support/openmw/screenshots/screenshot043.png) vs [`screenshot040.png`](file:///Users/stellz/Library/Application Support/openmw/screenshots/screenshot040.png)).
### Overview & Problem
In [`files/shaders/compatibility/terrain.frag`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/terrain.frag#L97-L137), single-pass terrain blends up to 6 splat layers.
For layers 1 through 5, the shader unconditionally samples the high-resolution diffuse map *before* checking the blend weight:
```glsl
#if @numLayers >= 2
vec4 diff1 = texture2D(diffuseMap1, adjustedUV);
float w1 = texture2D(blendMap1, blendMapUV).a;
blendedDiffuse += diff1.rgb * w1;
totalWeight += w1;
#endif
```
`diffuseMap1..5` are high-resolution textures (e.g., 1024×1024 or 2048×2048) requiring expensive anisotropic/trilinear filtering and texture cache lines. In contrast, `blendMap1..5` are low-resolution single-channel splat alpha masks.
In any given terrain cell, secondary layers (dirt paths, cobblestone roads, rock outcrops) only cover 10%–30% of the surface area. Over the remaining 70%–90% of the cell, `w1..w5` is `0.0`.
Unconditionally fetching all 5 diffuse textures forces the GPU to execute millions of wasted high-resolution texture reads per frame, thrashing the L1/L2 texture caches and wasting memory bandwidth on data that is subsequently multiplied by zero.
### Implemented Solution
Sample the low-resolution `blendMap` first. Only issue the high-resolution `diffuseMap` sample if the blend weight is non-zero:
```glsl
#if @numLayers >= 2
float w1 = texture2D(blendMap1, blendMapUV).a;
if (w1 > 0.001)
{
vec4 diff1 = texture2D(diffuseMap1, adjustedUV);
blendedDiffuse += diff1.rgb * w1;
totalWeight += w1;
}
#endif
```
### Empirical Hardware Results (8 Runs vs Control Baseline)
#### Pelagiad (`--timedemo=foliage`)
- **Control Baseline (`d1bd976ec6`, 8 Runs):** Mean **74.71 FPS** (±2.32), Frame Time **13.39 ms**, Draw **10.90 ms**. Runs: `[69.80, 73.70, 76.50, 76.50, 75.50, 73.60, 76.70, 75.40]`
- **Opportunity #23 (8 Runs):** Mean **76.42 FPS** (±1.39), Frame Time **13.09 ms**, Draw **10.89 ms**. Runs: `[76.10, 75.00, 73.90, 76.80, 76.70, 77.20, 78.10, 77.60]`
- **Delta:** **+1.71 FPS (+2.29%)**, **-0.30 ms Frame Time**, variance reduced by 40%.
#### Balmora (`--timedemo=balmora`)
- **Control Baseline (`d1bd976ec6`, 8 Runs):** Mean **56.14 FPS** (±1.02), Frame Time **17.81 ms**, GPU **23.40 ms**, Draw **16.84 ms**. Runs: `[56.00, 55.30, 57.00, 58.10, 54.90, 55.60, 56.30, 55.90]`
- **Opportunity #23 (8 Runs):** Mean **56.94 FPS** (±1.35), Frame Time **17.56 ms**, GPU **23.13 ms**, Draw **16.55 ms**. Runs: `[56.00, 56.50, 57.10, 55.70, 56.50, 57.90, 56.00, 59.80]`
- **Delta:** **+0.80 FPS (+1.42%)**, **-0.25 ms Frame Time**, **-0.27 ms GPU Timer**, peak **59.80 FPS**.
---
## 24. [FAILED / REJECTED] Pre-Fetch Distance Cutoff Discard in `groundcover.frag`
> [!CAUTION]
> **Status: Tested & Rejected (-2.0 FPS Regression in Pelagiad, -1.3 FPS in Balmora)**
> **Empirical Result (8-Trial Timedemo):** 74.36 FPS vs 76.42 FPS in Pelagiad (-2.7%); 55.67 FPS vs 56.94 FPS in Balmora (-2.2%).
> **Post-Mortem:** In [`files/shaders/compatibility/groundcover.vert`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/groundcover.vert#L146-L149), the vertex shader already culls all groundcover triangles beyond `@groundcoverFadeEnd` by collapsing `gl_Position` to a degenerate point:
> ```glsl
> if (length(gl_ModelViewMatrix * vec4(position, 1.0)) > @groundcoverFadeEnd)
> gl_Position = vec4(0.0, 0.0, 0.0, 1.0);
> else
> gl_Position = viewToClip(viewPos);
> ```
> The hardware primitive assembler / rasterizer culls degenerate zero-area triangles immediately before fragment shading. Consequently, **zero fragments ever reach `groundcover.frag` beyond `groundcoverFadeEnd`**. Adding `if (euclideanDepth >= @groundcoverFadeEnd) discard;` at the start of `groundcover.frag` was completely redundant and introduced dynamic branching divergence into every valid foreground fragment quad, hurting SIMD throughput. **Reverted immediately.**
### Overview & Problem
Groundcover batches are rendered in chunks whose spatial bounds often extend beyond the maximum fade distance. In `groundcover.frag`:
```glsl
if (euclideanDepth > @groundcoverFadeStart)
gl_FragData[0].a *= 1.0-smoothstep(@groundcoverFadeStart, @groundcoverFadeEnd, euclideanDepth);
gl_FragData[0].a = alphaTest(gl_FragData[0].a, alphaRef);
```
It was hypothesized that foliage fragments beyond `@groundcoverFadeEnd` were sampling `diffuseMap` unnecessarily prior to alpha testing.
### Empirical Post-Mortem & Architectural Findings
- **8-Run Benchmark Results:**
- Pelagiad: Dropped from **76.42 FPS** down to **74.36 FPS** (-2.06 FPS regression).
- Balmora: Dropped from **56.94 FPS** down to **55.67 FPS** (-1.27 FPS regression).
- **Root Cause:** Inspection of [`files/shaders/compatibility/groundcover.vert`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/groundcover.vert#L146-L149) revealed that the engine already collapses all triangle vertices beyond `@groundcoverFadeEnd` to a degenerate point `(0, 0, 0, 1)`. These triangles have zero pixel area and are eliminated by hardware rasterizers before any fragments are spawned.
- Reverting the discard check immediately restored performance to **75.97 – 76.80 FPS** in Pelagiad. The optimization is rejected.
---
## 25. [FAILED / REJECTED] Out-of-Volume Shadow Map Sampling Pruning in `shadows_fragment.glsl`
### Overview & Problem
In [`files/shaders/compatibility/shadows_fragment.glsl`](file:///Users/stellz/repos/openmw/files/shaders/compatibility/shadows_fragment.glsl#L38-L46):
```glsl
if (all(lessThan(shadowXYZ.xy, vec2(1.0, 1.0))) && all(greaterThan(shadowXYZ.xy, vec2(0.0, 0.0))))
{
shadowing = min(shadow2DProj(shadowTexture@shadow_texture_unit_index, shadowSpaceCoords@shadow_texture_unit_index).r, shadowing);
```
The cascade receiver shader currently checks only `shadowXYZ.xy` against the unit box `[0, 1]`.
It does not check `shadowXYZ.z`.
In an orthographic shadow projection (directional sunlight cascades), `shadowXYZ.z` represents normalized light-space depth. If a fragment lies in front of the shadow near plane (`shadowXYZ.z < 0.0`) or beyond the cascade's far plane (`shadowXYZ.z > 1.0`), it is completely outside the shadow frustum.
Because `shadowXYZ.z` is unchecked, fragments that fall within the 2D footprint but outside the light's depth bounds still issue the expensive `shadow2DProj` hardware PCF comparison texture fetch.
### Proposed Solution
Expand the frustum bounds test from `vec2` to `vec3` to include depth:
```glsl
if (all(lessThan(shadowXYZ, vec3(1.0))) && all(greaterThan(shadowXYZ, vec3(0.0))))
{
shadowing = min(shadow2DProj(shadowTexture@shadow_texture_unit_index, shadowSpaceCoords@shadow_texture_unit_index).r, shadowing);
```
### Empirical Validation & Test Results (8 Trials)
- **Status:** **Tested & Abandoned (Ineffective / Regressive)**
- **Baseline (Commit `baa9fe40d8`):**
- Pelagiad (`foliage`): Mean **76.42 FPS** (±1.39), Frame Time **13.09 ms**, GPU Timer **17.15 ms**
- Balmora (`balmora`): Mean **56.94 FPS** (±1.35), Frame Time **17.58 ms**, GPU Timer **23.13 ms**
- **Opportunity #25 (8-Run Benchmark):**
- Pelagiad (`foliage`): Mean **74.53 FPS** (±1.05) (**-1.89 FPS / -2.47%**), Frame Time **13.41 ms** (+0.32 ms), GPU Timer **16.90 ms**
- Balmora (`balmora`): Mean **55.91 FPS** (±1.91) (**-1.03 FPS / -1.8%**), Frame Time **17.90 ms** (+0.34 ms), GPU Timer **23.18 ms**
### Root Cause Analysis
1. **Prior View-Distance Cascade Partitioning:**
Due to the cascade split distance test implemented in Opportunity #12 (`distance <= cascadeSplits[@shadow_texture_unit_index]`), fragments are already screened along camera view distance before cascade evaluation. Consequently, virtually all visible fragments falling within the 2D $(x, y) \in [0, 1]^2$ shadow map footprint already lie within the light's depth range $[0, 1]$.
2. **Added Vector Comparison Overhead:**
Extending the vector comparison from `vec2` to `vec3` introduced extra scalar comparison instructions and condition evaluation into the inner shadow loop for every fragment on screen, while pruning virtually zero real `shadow2DProj` fetches.
3. **Conclusion:**
The check costs more in fragment ALU comparison instructions than it saves in avoided texture samples. Reverted immediately to preserve baseline performance.
---
## 26. Texture/Attribute Identity Canonicalization (Shared `osg::Texture2D` / `TextureType` StateAttributes)
> [!CAUTION]
> **Status: REJECTED / CLOSED — Executed Diagnostic (Opp #27) Found No Headroom. Do Not Implement.**
> OSG 3.6.5 source review invalidated the original StateGraph-merge rationale (see "Corrected Mechanism"), and the #27 diagnostic measured only **1 image (Balmora) / 2 images (Pelagiad)** bound by more than one distinct `osg::Texture2D` pointer out of ~294/~213 unique images. `osgDB::SharedStateManager` already unifies texture attributes across templates, so there is essentially no population for interning to canonicalize. **#26a and #26b are rejected.** The sub-step writeups are retained below as a post-mortem record.
### Original Hypothesis (Superseded)
The original motivation was: Opportunity #17 established that Balmora's main camera is **CPU Draw Traversal bound (~16 ms/frame under 1,645 drawables expanding into 509 StateGraphs)**; Opportunity #18 attacked this by interning `SceneUtil::Material` uniforms; the analogous texture case appeared unaddressed because [`components/nifosg/nifloader.cpp`](file:///Users/stellz/repos/openmw/components/nifosg/nifloader.cpp#L1136) `attachTexture()` allocates a fresh `osg::Texture2D` every call:
```cpp
static osg::ref_ptr attachTexture(...)
{
osg::ref_ptr texture2d = new osg::Texture2D(image); // <-- fresh heap object every call
// ...
stateset->setTextureAttribute(texUnit, texture2d, osg::StateAttribute::ON);
osg::ref_ptr textureType = new SceneUtil::TextureType(name);
textureType = shareAttribute(textureType); // <-- TextureType IS interned here
stateset->setTextureAttribute(texUnit, textureType, osg::StateAttribute::ON);
return texture2d;
}
```
The same fresh-`Texture2D` pattern exists in [`components/shader/shadervisitor.cpp`](file:///Users/stellz/repos/openmw/components/shader/shadervisitor.cpp#L363) for normal maps and line ~407 for specular maps.
### Corrected Mechanism (From OpenSceneGraph 3.6.5 Source Review)
The original writeup asserted that distinct-but-equal `Texture2D` pointers "branch into separate `StateGraph` nodes and inflate the 509 StateGraph transition count." **Direct reading of OSG 3.6.5 source shows this is wrong.** The corrected facts are:
1. **StateGraphs are keyed by `StateSet*` pointer, not by attribute contents.** In `include/osgUtil/StateGraph`:
```cpp
typedef std::map< const osg::StateSet*, osg::ref_ptr > ChildList;
// ...
inline StateGraph* find_or_insert(const osg::StateSet* stateset)
{
ChildList::iterator itr = _children.find(stateset);
if (itr!=_children.end()) return itr->second.get();
// ...
}
```
Two *distinct `StateSet` objects with identical contents* still produce **two separate StateGraphs**. Interning a `Texture2D` does **not** by itself merge StateGraphs — only sharing the whole `StateSet` object would. (The `compareAttributeContents` value-comparison branch of `StateSet::compare` is not what drives StateGraph grouping during cull.)
2. **The real payoff of interning is skipping redundant `StateAttribute::apply()` calls during Draw Traversal.** In `include/osg/State`:
```cpp
inline bool State::applyAttribute(const StateAttribute* attribute, AttributeStack& as)
{
if (as.last_applied_attribute != attribute) // RAW POINTER comparison
{
as.last_applied_attribute = attribute;
attribute->apply(*this); // issues glBindTexture + glTexParameteri etc.
// ...
return true;
}
else
return false;
}
```
When consecutive drawables present the **same texture pointer**, the entire `Texture::apply` / `applyTexParameters` sequence (texture-object lookup, several `glTexParameteri`, potential upload path) is skipped. In effect, interning turns repeated *value-equal but pointer-distinct* textures into pointer-equal ones, eliminating redundant per-draw GL state application. This is the **same mechanism** Opportunity #18 exploited for material uniforms — but it is a *state-application* saving, **not** a StateGraph-count saving.
3. **`osgDB::SharedStateManager` may already perform this unification.** OpenMW constructs `SharedStateManager` with the default `SHARE_ALL` mode ([`components/resource/scenemanager.cpp`](file:///Users/stellz/repos/openmw/components/resource/scenemanager.cpp#L495)) and invokes it via `SHARE_DUPLICATE_STATE` (optimize path) or `shareState()` (non-optimize path). `SharedStateManager::find(osg::StateAttribute*)` looks up a `std::set` whose comparator is by `compare()` **value**, and `shareTextures()` rewrites the `StateSet`'s texture attribute to the shared instance. Because `ShaderVisitor` and the NIF loader both run **before** `SharedStateManager::share()` in [`SceneManager::getTemplate`](file:///Users/stellz/repos/openmw/components/resource/scenemanager.cpp#L1000), the `Texture2D` objects from both 26a and 26b call sites are *already present* when `share()` runs and **should already be unified across templates**. Therefore the premise that these textures remain unshared is **unverified and probably at least partly false**.
### Genuinely Unshipped Fragment Found In This Area
One real gap survived the source review: the `SceneUtil::TextureType` labels created in [`components/shader/shadervisitor.cpp`](file:///Users/stellz/repos/openmw/components/shader/shadervisitor.cpp#L377) (normal/height map) and line ~422 (specular map) are constructed with `new` and **never interned**:
```cpp
writableStateSet->setTextureAttribute(unit,
new SceneUtil::TextureType(normalHeight ? "normalHeightMap" : "normalMap"),
osg::StateAttribute::ON);
```
The NIF loader interns its `TextureType` through `shareAttribute()` ([`nifloader.cpp:1160`](file:///Users/stellz/repos/openmw/components/nifosg/nifloader.cpp#L1160)), but these two sites do not. `TextureType::compare` includes `_name`, and it is a texture attribute (so it participates in the same `applyAttribute` pointer checks). These labels are always drawn from a tiny fixed string set (`"normalMap"`, `"normalHeightMap"`, `"specularMap"`), so a handful of shared static instances would be trivially correct. Impact is nevertheless expected to be small relative to diffuse textures, and must be measured.
### Relationship to Rejected Opportunity #19 (Do Not Conflate)
Opportunity #19 (StateSet flattening) was rejected because pushing StateSets down to leaf geometries created **distinct `StateSet` pointers for equal state**, which fractured `LessGeometry`'s raw-pointer-based geometry merging. Canonicalizing *attributes* is the opposite operation (decreasing pointer diversity for equal state), which — where it applies — **helps** `LessGeometry` and `StateSet::compare`. It does **not**, however, merge StateGraphs (see corrected mechanism above).
### Proposed Solution
Extend the existing `shareAttribute()` interning pattern to `osg::Texture2D` (and, for 26b, `SceneUtil::TextureType`), keyed so that two attributes with `compare() == 0` return the same pointer. This does **not** touch uniforms, StateSet hierarchy, or the render bins.
**However, because `SharedStateManager` may already cover the texture cases and the true payoff is `applyAttribute` skipping rather than StateGraph merging, both sub-steps must be gated behind the #27 diagnostic.** Do not implement blind.
### Sub-Step 26a — NIF World Textures (`nifloader.cpp` `attachTexture`)
- **Scope:** The diffuse/world textures loaded for NIF models — static architecture, clutter, terrain assets, actors.
- **Files:** [`components/nifosg/nifloader.cpp`](file:///Users/stellz/repos/openmw/components/nifosg/nifloader.cpp#L1136) (`attachTexture`; `attachNiSourceTexture`, `attachExternalTexture` funnel through it).
- **Corrected measurement target:** **Count of redundant `Texture::apply` invocations per frame** (not StateGraph count — interning does not change StateGraph count). Also measure Draw Traversal ms, draw-call count, GPU timer. Multi-trial (≥8) against the current baseline.
- **Precondition:** #27 diagnostic must show a meaningful population of *distinct `Texture2D` pointers per equal image* **after** `SharedStateManager` has run. If `SharedStateManager` already unifies them, 26a has no headroom and should be dropped.
- **Risk:** Low functionally, but likely **redundant**. Do not assume a win.
- **Commit:** Independent commit; independently revertable.
### Sub-Step 26b — Shader Visitor Normal/Specular Maps & `TextureType` Labels (`shadervisitor.cpp`)
- **Scope:** The `Texture2D` wrappers for auto-detected normal/specular maps (same `SharedStateManager` caveat as 26a) **plus the genuinely un-interned `TextureType` label attributes** — the one clearly unshipped fragment.
- **Files:** [`components/shader/shadervisitor.cpp`](file:///Users/stellz/repos/openmw/components/shader/shadervisitor.cpp#L363) (normal map), line ~407 (specular map), and the `new SceneUtil::TextureType(...)` sites at lines ~377 and ~422.
- **Corrected measurement target:** Same as 26a (redundant `apply()` calls), plus number of distinct `TextureType` instances by name. Because the affected population is small in stock Morrowind, expect a small-to-noise delta.
- **Precondition:** Same as 26a.
- **Risk:** Low. Verify no shader-state behavior change.
- **Commit:** Independent commit; independently revertable. Do **not** bundle with 26a.
### Why Two Commits (Not One)
Separated because: (1) different measurable populations (world textures vs normal/specular/labels); (2) different risk surfaces (object-paging/`LessGeometry` merge path vs shader-state path); (3) revertability — the scorecard requires each entry to stand on its own commit hash and measured delta (cf. #18, #23); (4) consistency with the one-mechanism-per-commit precedent.
### Expected Impact (To Be Verified — Premise Is Now Doubtful)
- **Corrected subsystem impact:** Fewer redundant texture-attribute `apply()` calls during Draw Traversal (if any remain after `SharedStateManager`). **No StateGraph-count reduction is expected from interning.**
- **Estimated End-to-End:** Unknown. The `SharedStateManager` overlap means this may be neutral. Per this project's history, StateGraph-adjacent hypotheses have ranged from wins (#18) to noise (#14) to regressions (#19), so this is treated strictly as a hypothesis.
- **Implementation Complexity:** Very Low – Low.
- **Success Criterion:** Measured reduction in redundant `apply()` calls or Draw Traversal ms in Balmora, with pixel-identical rendering (screenshot diff) and no uptick in geometry-fracture draw calls. **If #27 shows no applicable population, formally reject/close all of #26 without implementing.**
---
## 27. [DIAGNOSTIC] StateSet / Texture Identity Profiling Instrumentation
> [!NOTE]
> **Status: COMPLETED — Instrumentation Implemented and Executed. Verdict recorded at the end of this section: #26a/#26b rejected.**
### Overview & Problem
Every StateGraph-reduction effort so far (#18, #19, #26) has been formulated against the single aggregate number from Opportunity #17 — "506 StateGraphs across 235 unique StateSets" — **without knowing what those StateSets actually differ by, or how OSG turns them into StateGraphs.** We are optimizing blind:
- #18 *assumed* material uniforms were a major contributor (it happened to be right).
- #19 *assumed* intermediate node StateSets were the problem (it was wrong, and made things worse).
- #26 *assumed* un-shared `Texture2D` wrappers both persist and matter. OSG 3.6.5 source review (see #26 "Corrected Mechanism") showed the StateGraph-merge claim is false, and the texture-unification claim is probably already handled by `osgDB::SharedStateManager`. **The #26 premise is now doubtful and must be measured before any code is written.**
We need authoritative data on two orthogonal questions:
**Q1 — StateSet composition:** Of the ~235 unique StateSets, how many differ only by (a) texture attributes, (b) material uniforms, (c) blend/depth/alpha modes, (d) shader program, (e) render bin, etc.? (Recall StateGraphs are keyed by `StateSet*`, so only *sharing whole StateSets* merges StateGraphs.)
**Q2 — Texture/attribute identity:** After `osgDB::SharedStateManager` has run during template loading, how many **distinct `osg::Texture2D` pointers exist per unique `osg::Image`**? Equivalently, how many redundant `Texture::apply` / `applyTexParameters` invocations occur per frame (the actual cost interning would remove)? This directly answers whether #26a/#26b have any headroom at all.
### Proposed Solution
Extend the existing timedemo instrumentation (which already records per-frame visible drawable counts — added in [`21d698ea55`](file:///Users/stellz/repos/openmw)) to, when an env var (e.g. `OPENMW_STATESET_DUMP=`) is set, walk the main-camera `RenderStage` StateGraph tree for one representative frame and emit a report:
1. Total `StateGraph` count and unique `StateSet` count.
2. For each unique `StateSet`, a category histogram of the differing attribute/uniform members (Q1).
3. A ranked list of the most common "StateSet fingerprints" by drawable count, showing which attributes would need canonicalizing to collapse the largest groups (Q1).
4. **Texture identity census (Q2):** map each `osg::Image*` to the set of distinct `Texture2D*` bound to it; report counts of images with >1 distinct texture pointer, plus a per-frame tally of `Texture::apply` invocations split into "pointer changed" (applied) vs "pointer unchanged" (skipped).
This uses the already-present `RenderLeafTraverser` / `osgUtil::StateGraph` traversal machinery in [`components/sceneutil/mwshadowtechnique.cpp`](file:///Users/stellz/repos/openmw/components/sceneutil/mwshadowtechnique.cpp#L178) as a starting point, or a small dedicated visitor.
### Why This Comes First
1. **Prevents repeating the #19 mistake.** #19 was implemented on an unverified StateSet assumption and regressed. #26 is currently in the same position, and source review has already falsified part of its rationale.
2. **It directly decides #26's fate.** If the census shows `SharedStateManager` already unifies textures (few images with >1 texture pointer, few redundant `apply` calls), **#26a and #26b should be formally rejected/closed without implementation** — saving commit budget. If a meaningful population remains, the census tells us *which* sites (26a vs 26b vs the un-interned `TextureType` labels) to target and gives a baseline to measure against.
3. **It scopes the StateGraph track generally.** Q1's histogram tells us whether further StateGraph/StateSet work is worth pursuing at all, or whether Balmora's ~16 ms Draw Traversal is dominated by something else entirely.
### Results (Executed)
> [!IMPORTANT]
> **Implemented and run. Verdict: #26a and #26b are REJECTED (no headroom). Recorded here as reference data.**
Implementation lives in [`apps/openmw/statesetprofiler.{hpp,cpp}`](file:///Users/stellz/repos/openmw/apps/openmw/statesetprofiler.cpp), invoked from [`apps/openmw/engine.cpp`](file:///Users/stellz/repos/openmw/apps/openmw/engine.cpp) after the timedemo measured window when `OPENMW_STATESET_DUMP` is set. It traverses `viewer->getSceneData()` (the loaded scene graph, deterministic, independent of camera movement) and reports Q1 and Q2.
**Reproduction:**
```sh
OPENMW_STATESET_DUMP=1 ./openmw --skip-menu --no-sound --timedemo balmora --timedemo-frames 10 --timedemo-warmup 2
```
**Balmora (Apple Silicon, three runs):**
| Metric | Value |
| :--- | :--- |
| Unique `StateSet` objects | 328–329 |
| Node references to them | 1,964 |
| Distinct content fingerprints (Q1) | **35** |
| `Texture2D` bindings across unique StateSets | 313 |
| Unique images referenced | 293–294 |
| **Images bound by >1 distinct `Texture2D` ptr (Q2)** | **1** |
| Max distinct `Texture2D` ptrs for one image | 2 |
**Pelagiad (one run):** 259 unique StateSets, **38** fingerprints, 252 texture bindings, 213 unique images, **2** images bound by >1 distinct `Texture2D` ptr.
**Top fingerprint groups (Balmora):**
```
[104 StateSets / 936 refs] texA=1;texM=0;attr=2;mode=1;uni=8;bin=0:0:;
[158 StateSets / 641 refs] texA=1;texM=0;attr=1;mode=1;uni=0;bin=0:0:;
[ 3 StateSets / 110 refs] texA=0;texM=0;attr=1;mode=0;uni=0;bin=0:0:;
```
### Conclusion & Verdict
1. **#26a and #26b: REJECTED / CLOSED without implementation.** Q2 shows that of ~294 unique images, only **1** is bound by more than one distinct `osg::Texture2D` pointer in Balmora (2 in Pelagiad). `osgDB::SharedStateManager` (constructed with `SHARE_ALL` and run during `SceneManager::getTemplate`) has **already unified texture attributes across templates**. There is effectively **no population for texture interning to canonicalize**, so the hypothesized `State::applyAttribute` savings do not exist. Implementing #26a/#26b would be redundant work for a ~1-image delta. This confirms, with data, the doubt raised by the OSG source review — and avoids repeating the #19 mistake of optimizing an unverified assumption.
2. **Where the StateSets actually differ (Q1):** the 328–329 unique StateSets collapse to only **35 content fingerprints**. The two dominant patterns (`texA=1;attr=2;uni=8` and `texA=1;attr=1;uni=0`) cover 262 StateSets / 1,577 node references. So the vast majority of StateSets are **value-identical** and differ only in *pointer identity*. Because OSG keys `StateGraph` children by `StateSet*` pointer (see #26 corrected mechanism), the remaining fragmentation is a **whole-`StateSet` sharing** problem, **not** an attribute-interning problem. Any future StateGraph work must target canonicalizing/sharing entire `StateSet` objects — which is precisely the direction Opportunity #19 attempted and which fractured `LessGeometry`'s raw-pointer merge. This is therefore a **high-risk, high-complexity** track, and the data shows the *content* is already almost fully deduplicated.
3. **No further texture-attribute work is warranted.** The diagnostic has done its job: it closed the #26 track and redirected any residual StateGraph effort toward whole-StateSet canonicalization (with the #19 pitfalls as a known hazard).
---
## 28. [DIAGNOSTIC / COMPLETED] CPU Draw Traversal Profiler (Transitions & Per-Transition Payload)
> [!NOTE]
> **Status: COMPLETED — Instrumentation Implemented and Executed. Findings below drive Opportunity #29.**
### Motivation
Opportunity #17 established that Balmora is CPU **Draw Traversal** bound (~16 ms/frame), but the project had no runtime breakdown of *what* that time is spent on. Static scene-graph profiling (#27) showed 328 unique StateSets collapsing to 35 content fingerprints, but not how many times each is **applied per frame** nor what each application costs. Without that, any further CPU-side optimization would again be guesswork (the #19 failure mode).
Because OpenMW cannot modify OpenSceneGraph, but `osgUtil::RenderBin::drawImplementation` is **virtual** and RenderBin **prototypes are replaceable at runtime** (`osgUtil::RenderBin::addRenderBinPrototype`), we can wrap the unmodified OSG draw traversal from OpenMW's side.
### Implementation
[`apps/openmw/drawtraversalprofiler.{hpp,cpp}`](file:///Users/stellz/repos/openmw/apps/openmw/drawtraversalprofiler.cpp):
- Subclasses `osgUtil::RenderBin` (installed as the `"RenderBin"` prototype) and overrides `drawImplementation` to (a) time the unmodified base draw traversal, and (b) once per frame (keyed on the frame stamp) recursively inspect the whole `RenderStage` bin/StateGraph tree.
- **StateGraph transitions** are counted per `StateGraph` that has leaves (one state application, matching OSG's `moveStateGraph`/`State::apply` cadence). **Draw calls** are the total `RenderLeaf` count. **Per-transition payload** is measured per applied StateSet (texture units, texture attributes, non-texture attributes, modes, uniforms).
- A **uniform-name fingerprint** histogram groups applied StateSets by their sorted uniform names, ranked by total applications/frame.
- Installed from [`apps/openmw/engine.cpp`](file:///Users/stellz/repos/openmw/apps/openmw/engine.cpp) when `OPENMW_DRAW_PROFILE` is set; the accumulator is reset when timedemo warmup ends. Rendering path is unmodified; instrumentation only.
**Reproduction:**
```sh
OPENMW_DRAW_PROFILE=1 ./openmw --skip-menu --no-sound --timedemo balmora --timedemo-frames 30 --timedemo-warmup 10
```
### Results (Deterministic — identical across runs)
| Metric | Balmora | Pelagiad |
| :--- | :---: | :---: |
| Draw calls (main-camera RenderLeaves) | **780** | 425 |
| **StateGraph transitions / frame** | **506** | **178** |
| **Drawables per transition** | **1.54** | 2.39 |
| Texture units applied / frame | 617 | 402 |
| Texture attributes applied / frame | 1,056 | 508 |
| Non-texture attributes applied / frame | 953 | 362 |
| Modes applied / frame | 554 | 233 |
| **Uniforms applied / frame** | **2,841** | 1,041 |
| Texture-unit high-water mark | 16 | — |
**Validation:** 506 transitions/frame matches the previously documented "509 StateGraphs" for the main camera almost exactly, confirming the instrumentation measures the intended quantity.
### Uniform-Name Fingerprint Results (Balmora, ranked by applications/frame)
```
310 x texUnits=1 texAttrs=2 attrs=2 modes=1 uniforms=8
[material.ambient, material.diffuse, material.emission, material.emissiveMult,
material.shininess, material.specStrength, material.specular, material.vertexColorMode]
130 x texUnits=1 texAttrs=2 attrs=1 modes=1 uniforms=0 []
41 x texUnits=2 texAttrs=2 attrs=4 modes=2 uniforms=5 [blendMap, colorMode, diffuseMap, texMat0, texMat1]
7 x texUnits=1 texAttrs=1 attrs=1 modes=0 uniforms=2 [colorMode, diffuseMap]
7 x texUnits=4 texAttrs=4 attrs=2 modes=2 uniforms=7 [blendMap0, blendMap1, colorMode, diffuseMap0, diffuseMap1, texMat0, texMat1]
```
### Key Findings
1. **Balmora is in a state-fragmentation regime: ~1.54 drawables per state transition.** Nearly every draw call forces an OSG state change (`moveStateGraph` + `State::apply`). Pelagiad is ~2.4→ healthier. This is the concrete form of the CPU draw-traversal bottleneck.
2. **The single dominant cost is the 8 `material.*` uniforms, applied 310×/frame — 61% of all 506 transitions.** These are exactly the uniforms interned by Opportunity #18. #18 shared the `osg::Uniform` *pointers* (eliminating heap churn and enabling `StateSet::compare` matching), but they are **still re-applied per drawable StateSet** because they live on each drawable's own StateSet rather than on a shared ancestor. `State::apply`'s `applyUniformList` thus walks these 8 uniforms 310 times/frame.
3. **The second-largest fingerprint (`130 x, uniforms=0`) is texture-only state** (1 texture unit, 0 uniforms) — cheap per transition.
4. **Payload is uniform-dominated:** 2,841 uniforms/frame ÷ 506 transitions ≈ 5.6 uniforms/transition.
### Scope & Limitations
- The profiler captures the **first camera's RenderStage to draw each frame** (the main camera), which is the target of the documented "509 StateGraphs" metric. It does not aggregate shadow/reflection cameras.
- The reported `ms/frame` is the sum of time inside the instrumented bins' `drawImplementation`; it is a **lower bound** on total `sceneView->draw()` time (some render-stage leaf rendering bypasses the sub-bin prototypes). The **volume and payload metrics are the decision-relevant output** and are precise (validated against the known 509 figure).
### Verdict
This confirms the direction for the next optimization: **Candidate A — reduce per-transition payload, specifically the repeated application of the 8 `material.*` uniforms** — over Candidate B (whole-StateSet content-sharing), which carries the #19 hazard. See Opportunity #29.
---
## 29. Hoisting Shared `material.*` Uniforms out of Per-Drawable StateSets
> [!NOTE]
> **Status: Proposed — Identified by #28. Not yet implemented.**
### Overview & Problem
Opportunity #28 measured that the **8 `material.*` uniforms are applied 310 times per frame in Balmora — 61% of all 506 state transitions**. These uniforms (`material.ambient`, `material.diffuse`, `material.emission`, `material.emissiveMult`, `material.shininess`, `material.specStrength`, `material.specular`, `material.vertexColorMode`) are set by [`SceneUtil::Material::setStateSet`](file:///Users/stellz/repos/openmw/components/sceneutil/material.cpp) on each drawable's own StateSet.
Opportunity #18 already interned the `osg::Uniform` **pointers** so that (a) heap allocation is avoided and (b) OSG's `State::applyAttribute`/`applyUniform` pointer-equality check can skip re-application when the immediately-previous draw used the same pointer. However, the diagnostics show the material uniforms are **still applied 310×/frame**: they live on per-drawable StateSets whose identity changes frequently enough (interleaved with texture state) that `State::apply` re-walks and re-applies them. In Morrowind, the overwhelming majority of static architecture shares identical material values (diffuse/ambient `(1,1,1,1)`, specular/emission near default), so re-applying them per drawable is wasted work.
### Proposed Solution (to be validated)
Reduce the **number of StateSets that carry material uniforms**, so `State::apply` handles them once per group rather than once per drawable:
1. **Identify whether the material uniforms are actually equal across the 310 applications.** Extend the #28 profiler (or reuse the #27 fingerprint) to report the distinct *values* of the material uniforms across applied StateSets; if a small number of value-sets dominate (likely one or two), hoisting is worthwhile.
2. **Hoist the common material values to a shared ancestor StateSet** where they are uniform across a subtree (e.g. the static-architecture container), with `osg::StateAttribute::OVERRIDE`/`PROTECTED` semantics so per-drawable exceptions can still override. Then `StateGraph` push/pop applies the material once for the subtree, and per-drawable StateSets no longer carry them.
3. **Fallback if a large fraction differ:** leave per-drawable material StateSets but ensure they are the *only* difference (so StateGraph grouping can keep the material-heavy drawables adjacent).
This is the safe analogue of #18 for the *placement* of the uniforms rather than their storage.
### Risk & Constraints
- Must not change rendered output: material values, override precedence, and animated material controllers (see `MaterialColorController` in [`nifloader.cpp`](file:///Users/stellz/repos/openmw/components/nifosg/nifloader.cpp)) must keep working. Animated materials that modify `material.*` per frame cannot be hoisted.
- Interacts with object paging / `LessGeometry` merging: hoisting must not fracture geometry batches (the #19 hazard). Any hoisting to ancestor StateSets must keep content-equal StateSets pointer-equal, not the reverse.
- Must be measured with the #28 profiler (transitions/frame and uniforms/frame should drop) plus timedemo Draw Traversal ms and screenshot pixel-diff.
### Expected Impact
- **Target:** reduce the 310×/frame `material.*` applications substantially, cutting `State::apply` uniform-walk work and the per-transition payload.
- **Estimated End-to-End:** to be measured; the project's history (wins #18, noise #14, regressions #19/#20/#25) means this is treated strictly as a hypothesis with a clear profiler-observable success metric.
- **Implementation Complexity:** Low – Medium.
- **Success Criterion:** `material.*` applications/frame drop significantly in the #28 profile; Balmora Draw Traversal ms decreases; pixel-identical output; no draw-call increase.
---
## Comparative Impact & Effort Matrix
| Rank | Optimization | Estimated End-to-End Frame Time Gain | Subsystem Savings | Primary Scenario Where Observed | Implementation Complexity | Status / Result |
|:----:|--------------|:-------------------------------------:|:-----------------:|---------------------------------|:-------------------------:|:---------------:|
| **—** | **Progressive Cascade Small-Feature Culling** | **+16.3% (+7.2 FPS)** *(Verified)* | 307 draw calls/frame eliminated; -15.3% GPU time | All exterior scenes with shadows | Low (Bugfix + Tuning) | **Completed (`dee26420f3`)** |
| **—** | **Resolution-Scaled Camera Small-Feature Culling** | **+18.4% (+9.2 FPS)** *(Verified)* | Cull subpixel/tiny clutter scaled to 1080p; -13.5% GPU time, -17.9% Draw time | High-res/Retina exterior scenes | Low | **Completed (`87ad857410`)** |
| **—** | **Interior Static Architecture Batching** | **Neutral** *(Verified)* | 66% static node reduction; interior draw consolidation | Interior cells with static architecture | Low – Medium | **Completed (`6426c62780`)** |
| **—** | **Per-Frame Invariant Hoisting in `fog.glsl`** | **+8.4% (+4.5 FPS)** *(Verified)* | -1.59 ms GPU time; eliminates 4x4 matrix multiply per fragment | Universal across all passes | Very Low | **Completed (`6d8a54cce2`)** |
| **—** | **Single-Pass Multi-Texture Terrain Splatting** | **+7.5% FPS** *(Verified)* | 50%–75% terrain draw calls eliminated; -0.96 ms frame time in wilderness | Exterior scenes with terrain | Medium | **Completed (`ff6bfa4657`)** |
| **—** | **Linear Depth Cascade Split Early-Out & Culling Fix** | **-1.11 to -1.53 ms GPU** *(Verified)* | Peak 79.8 FPS (+2.4 FPS); skips earlier cascades & fixes distance culling | Wilderness and foliage vistas | Low | **Completed (`c56f296a98`)** |
| **—** | **Shared Shadow Receiver StateSet & RenderStage Hoisting** | **-0.4 ms Draw / -1.02 ms GPU** *(Verified)* | Prunes intermediate StateGraph layer across all drawables | Universal across scenes with shadows | Low | **Completed (`7bb3541b65`)** |
| **—** | **Universal Directional Light & Shader Math Invariants** | **0.0% FPS gain** *(Tested)* | Pure noise; ALU hidden behind memory/texture fetch latency | Universal across all exterior & interior scenes | Very Low | **Tested / Ineffective (Eliminated)** |
| **—** | **Front-to-Back Sorting for Opaque RenderBin** | **-11.0% FPS** *(Tested)* | Broken state batching; TBDR HSR renders redundant | High-res/MSAA in dense settlements | Low | **Tested / Ineffective (Eliminated)** |
| **—** | **Tuned Object Paging Merge Factor (500)** (Opp #15) | **+1.0 to +3.2 FPS in Balmora** *(Verified)* | Eliminates 36 draw calls/frame; Active-grid spatial batching failed (+1,800 calls) & reverted | Dense towns and exterior settlements | Low (Tuning) | **Completed (`4d8ecf6435`, `21d698ea55`)** |
| **—** | **Material Uniform Deduplication & Shared Uniform Caching** (Opp #18) | **-0.45 ms Draw / -0.48 ms GPU** *(Verified)* | Peak 60.4 FPS in Balmora; collapses uniform allocations & enables pointer matching | Dense settlements (Balmora) | Low | **Completed (`d1bd976ec6`)** |
| **—** | **StateSet Flattening & Hierarchy Consolidation** (Opp #19) | **+6.6% Draw Calls (+196 calls)** *(Tested)* | Pointer segregation in `LessGeometry` broke exterior chunk batching | Exterior settlements (Balmora, Pelagiad) | Medium | **Tested / Ineffective (Abandoned)** |
| **—** | **Zero-Weight Diffuse Fetch Elimination in Single-Pass Terrain** (Opp #23) | **+2.3% FPS in Pelagiad (+1.71 FPS)** *(Verified)* | Skips 30%–60% of high-res diffuse fetches; 8-trial verified mean 76.4 vs 74.7 FPS, -0.30 ms Frame Time | Landscape vistas (Pelagiad) | Very Low | **Completed (`baa9fe40d8`)** |
| **—** | **Pre-Fetch Distance Cutoff Discard in `groundcover.frag`** (Opp #24) | **-2.0 FPS Regression** *(Tested)* | Triangles beyond fade radius are already collapsed to zero area in vertex shader; discard branch caused SIMD quad divergence | Exterior scenes with groundcover | Very Low | **Tested / Rejected (Reverted)** |
| **—** | **Rasterizer Interpolator Pruning (`normalToViewMatrix`)** (Opp #20) | **-2.0 to -2.4 FPS Regression** *(Tested)* | Driver cross-stage DCE was already pruning dead varyings; dynamic uniform math added ALU overhead | Universal across objects, terrain, groundcover | Very Low | **Tested / Rejected (Reverted)** |
| **—** | **Out-of-Volume Shadow Map Sampling Pruning** (Opp #25) | **-1.0 to -1.9 FPS Regression** *(Tested)* | View-distance splits already partition depth; 3D vector comparison added ALU instruction overhead | Exterior scenes with shadows | Very Low | **Tested / Rejected (Reverted)** |
| **—** | **Texture/Attribute Identity Canonicalization — NIF World Textures** (Opp #26a) | **REJECTED (no headroom)** | Diagnostic #27 measured only 1 image / 294 bound by >1 distinct `Texture2D`; `SharedStateManager` already unifies textures | Dense settlements (Balmora) | Very Low – Low | **Rejected / Closed (never implemented)** |
| **—** | **Texture/Attribute Identity Canonicalization — Normal/Specular Maps & `TextureType` Labels** (Opp #26b) | **REJECTED (no headroom)** | Same as 26a; no applicable `Texture2D` population found | Assets with normal/specular maps | Very Low – Low | **Rejected / Closed (never implemented)** |
| **—** | **StateSet / Texture Identity Profiling Instrumentation** (Opp #27) | **None (diagnostic, executed)** | Measured 328 StateSets → 35 fingerprints; 1 image with >1 texture ptr. Closed the #26 track | Dense settlements (Balmora, Pelagiad) | Low | **Completed (instrumentation)** |
| **—** | **CPU Draw Traversal Profiler** (Opp #28) | **None (diagnostic, executed)** | Measured 506 transitions/frame, 1.54 drawables/transition, 2,841 uniforms/frame; isolated the 8 `material.*` uniforms at 310 applications/frame | Dense settlements (Balmora) | Low | **Completed (instrumentation)** |
| **1** | **Hoist Shared `material.*` Uniforms out of Per-Drawable StateSets** (Opp #29) | **To be measured** | Targets the 310×/frame `material.*` applications found by #28 (61% of transitions); reduce per-transition uniform payload | Dense settlements (Balmora) | Low – Medium | **Proposed — Top CPU Candidate** |
| **1** | **Early-Z & Overdraw Mitigation for Foliage (RenderBin Ordering)** (Opp #4) | **10% – 25%** | RenderBin 1 ordering allows solid terrain/architecture to write depth first, unlocking Early-Z rejection of grass | Bitter Coast, dense forests, grass vistas | Very Low – Low | **High Priority (Top Candidate)** |
| **2** | **Dead Math & Redundant Normalization in `objects.frag`** (Opp #21) | **1% – 2% GPU** | Eliminates dead screen-space division, 3D normalization, and viewNormal ops | Universal across static & dynamic objects | **Very Low** | Quick Win Candidate |
| **3** | **Water Wave Normal Texture Latency Decoupling** (Opp #8) | **3% – 7%** | Decouples 6 serialized dependent texture fetches into parallel lookups | Coastal viewpoints and open sea | **Low** | Medium Priority (when water shaders on) |
| **4** | **Defaulting to Clustered Lighting (SSBO)** (Opp #5) | **10% – 20%** | 40% – 70% of light pass | Dungeons, tombs, night settlements | Low (config / tuning) | Engine Feature |
| **5** | **Optimizing Cascaded Shadow Mapping: View-Depth Selection vs. Per-Cascade Varyings** (Opp #7) | **3% – 8% GPU** | Selects cascade in fragment shader; frees 4–8 vec4 interpolator registers | All exterior scenes with shadows | Medium | Architectural Project |
| **6** | **Hardware GPU Vertex Skinning** (Opp #6) | **5% – 15%** | 100% of dynamic VBO uploads | NPC-heavy towns (Balmora, Vivec) | Medium – High | Major Architectural |
| **7** | **Vertex Shadow Offset Hoisting in `shadows_vertex.glsl`** (Opp #22) | **<1% Vertex ALU** | Hoists normal offset math outside 4-cascade unrolled loop | All scenes with shadows | **Very Low** | Low Priority |
| **8** | **Frustum-Culling Water Reflection RTT Pass** (Opp #3) | **15% – 35%** (situational) | 100% of reflection pass | Facing away from water (when water shaders on) | Low – Medium | On Hold (`[Water] shader = false`) |