# Animation System Design

**Component:** Animation System (Rust/Fyrox)
**Status:** MVP Implemented (placeholder cube), Full Design Pending
**Related:** [Frontend Architecture](./frontend-architecture.md), [IPC Protocol](./ipc-protocol.md)

---

## Implementation Status

**Current MVP** (`game/src/animation_state.rs`, `game/src/animation_events.rs`, `game/src/cube_animator.rs`):
- ✅ `BodyState` enum with all states (Idle, Working, TurningToCamera, Talking, TurningFromCamera)
- ✅ `IdleVariant` enum (Sitting, Thinking, Stretching, LookingAround, Wandering)
- ✅ `WorkAnimation` enum (Terminal, Reading, Writing, WebBrowsing, DeepThinking)
- ✅ `AnimationController` with state transitions
- ✅ Idle variation timer (10-30 second random changes)
- ✅ Tool categorization for 15+ tool types
- ✅ Backend message integration (ToolStart, ToolEnd, AssistantTextDelta, AssistantMessageComplete)
- ✅ Color-coded placeholder cube for visual feedback
- ✅ **Event-driven turn animations** via convergence detection (replaces fixed timers!)
- ✅ **Animation event system** (`AnimationEvent`, `AnimationEventSource` trait) - Fyrox-idiomatic API
- ✅ Camera focus controller (`game/src/camera_animator.rs`)
- ✅ **Sound effects system** (`game/src/audio/mod.rs`) - Mumble beeps during speaking, paper rustle for notes

**Not Yet Implemented:**
- Fyrox animation machine with skeletal animations
- Blend shape system for facial expressions
- 3D character model

---

## Event-Driven Animation Architecture (MVP)

The MVP uses **convergence-based event detection** instead of fixed timers for turn animations.
This is more accurate and prepares for future Fyrox Animation Machine integration.

### How It Works

```
┌─────────────────────┐
│ AnimationController │ ← Decides state (TurningToCamera, Talking, etc.)
└──────────┬──────────┘
           │ state query
           ▼
┌─────────────────────┐
│    CubeAnimator     │ ← Interpolates position/rotation, detects convergence
└──────────┬──────────┘
           │ AnimationEvent (when rotation converges)
           ▼
┌─────────────────────┐
│ AnimationController │ ← Processes events, transitions state
└─────────────────────┘
```

### Event Types

| Event | Trigger | Action |
|-------|---------|--------|
| `turn_to_camera_complete` | Rotation converges to camera-facing | TurningToCamera → Talking |
| `turn_from_camera_complete` | Rotation converges to forward-facing | TurningFromCamera → Idle/Working |

### Why Event-Driven?

**Old approach (fixed timers):**
```rust
if self.turn_timer >= 1.0 {  // Assumes 1 second turn
    self.body_state = BodyState::Talking;
}
```

**New approach (convergence detection):**
```rust
// CubeAnimator detects when rotation reaches target
if rotation_angle < 0.01 {  // Actually aligned
    events.push(AnimationEvent::new("turn_to_camera_complete"));
}
// AnimationController responds to event
fn process_events(&mut self, events: impl Iterator<Item = AnimationEvent>) {
    for event in events {
        if event.name == "turn_to_camera_complete" {
            self.body_state = BodyState::Talking;
        }
    }
}
```

**Benefits:**
1. Responds to actual animation completion, not assumed duration
2. Works correctly even if frame rate varies
3. Seamlessly migrates to Fyrox Animation Machine (just swap event source)

### Speaking Duration (Intentionally Timer-Based)

Speaking duration remains timer-based because it depends on **message length**, not animation completion:
- Calculated as `chars / 50.0` (50 characters/second reading pace)
- Clamped to 1-30 seconds
- When real lip-sync arrives, the talking animation will LOOP until this timer expires

---

## Asset Authoring Roadmap

The placeholder cube is **intentional** for MVP development. This allows the animation state machine and backend integration to be tested without artist dependency.

### Why Placeholder First?

1. **Decouple development**: Engineers can build animation logic while artists work on models
2. **Test state transitions**: Color-coded cube makes state changes visible and debuggable
3. **Iterate on timing**: Tune transition durations without reworking animation clips
4. **Reduce risk**: Prove the system works before investing in art assets

### Production Character Timeline

| Phase | Milestone | Description |
|-------|-----------|-------------|
| **Phase 1** (Current) | Placeholder Cube | Color-coded states, all animation logic implemented |
| **Phase 2** | Character Blockout | Basic mesh (~1K triangles) with skeleton, test rigging |
| **Phase 3** | Animation Clips | Author 13 animation clips in Blender |
| **Phase 4** | Character Final | Full detail mesh (~5K triangles), textures, blend shapes |
| **Phase 5** | Polish | Sound effects, particle effects, final tuning |

### Asset Specifications

**Character Model:**
- Low-poly humanoid mesh (~5K triangles)
- Victorian-era clothing aesthetic
- Neutral pose for rigging
- PBR materials (albedo, normal, roughness, metallic)

**Skeleton:**
- ~50 bones humanoid rig
- Standard naming convention for Fyrox import
- IK targets for hands and feet

**Facial Blend Shapes:**
- `jaw_open` - Talking
- `brow_down` - Thinking/concentration
- `smile` - Pleased expression
- `blink` - Eye blink

**Animation Clips (13 total):**
- See "Animation Clips" table below for full list
- Export as separate files or NLA strips in Blender
- Target ~200KB per clip (~2.6MB total)

### Asset Source Options

1. **Custom Commission**: Hire artist for bespoke Victorian character
2. **Mixamo Library**: Start with [Mixamo](https://mixamo.com) base, customize
3. **In-House**: Create in Blender (requires 3D art skills)
4. **Asset Store**: Purchase and modify existing character

**Recommendation**: Start with Mixamo base character to validate pipeline, then commission custom character for final release.

### Integration Steps

When ready to replace placeholder cube:

1. Export character from Blender as GLTF
2. Import into Fyrox editor
3. Set up animation machine with states matching `AnimationController`
4. Update `animation_state.rs` to target real skeleton/mesh handles
5. Add blend shape support to `FaceController`
6. Test all state transitions
7. Tune blend times and timing

---

## Overview

The animation system brings the AI agent character to life through context-aware animations that correspond to tool usage, conversation state, and subagent spawning. It uses Fyrox's built-in animation state machine to blend between animations and synchronize with backend events.

### Design Goals

1. **Responsive**: Animations triggered immediately on backend events (<100ms latency)
2. **Contextual**: Different animations for different tools (Bash vs Read vs WebSearch)
3. **Smooth**: Seamless blending between states (no jarring transitions)
4. **Performant**: 60fps with real-time skeletal animation
5. **Extensible**: Easy to add new animations for new tools

---

## Animation Assets

### Character Rig

- **Skeleton**: Humanoid rig (~50 bones)
- **Skinned mesh**: Low-poly character model (~5K triangles)
- **Authored in**: Blender, exported as FBX or GLTF

### Animation Clips

| Clip | Duration | Loop | Trigger |
|------|----------|------|---------|
| **idle_sitting** | 3s | Yes | Default state |
| **idle_thinking** | 4s | Yes | Random idle variation |
| **idle_stretching** | 5s | No | Random idle variation |
| **idle_looking_around** | 6s | Yes | Random idle variation |
| **idle_wandering** | 3s | Yes | Random idle variation (agent strolls around room) |
| **turn_to_camera** | 1s | No | Assistant message starts |
| **turn_from_camera** | 1s | No | Assistant message ends |
| **talking** | 2s | Yes | Talking (blended with face animation) |
| **typing_terminal** | 3s | Yes | Bash, Execute tools |
| **reading_document** | 3s | Yes | Read, Glob, Grep tools |
| **writing_document** | 3s | Yes | Write, Edit tools |
| **web_browsing** | 4s | Yes | WebSearch, WebFetch tools |
| **deep_thinking** | 5s | Yes | Task, Explore tools (subagent spawn) |
| **gesture_explaining** | 2s | No | Occasionally during talking |

**Total storage**: ~30 clips × ~200KB/clip = ~6MB

---

## State Machine

### State Diagram

```
┌─────────────────────────────────────────────────────────────┐
│                      Idle Layer                             │
│                                                             │
│  ┌──────────┐      ┌──────────┐      ┌──────────┐         │
│  │  Sitting │ ───► │ Thinking │ ───► │Stretching│         │
│  │          │ ◄─── │          │ ◄─── │          │         │
│  └──────────┘      └──────────┘      └──────────┘         │
│       │                  │                  │              │
│       └──────────────────┴──────────────────┘              │
│                          │                                 │
│                          ▼                                 │
│                 ┌─────────────────┐                        │
│                 │ Looking Around  │                        │
│                 └─────────────────┘                        │
│                                                             │
└─────────────────────────┬───────────────────────────────────┘
                          │
              tool_start event
                          │
                          ▼
┌─────────────────────────────────────────────────────────────┐
│                      Work Layer                             │
│                                                             │
│  ┌───────────┐   ┌──────────┐   ┌──────────┐              │
│  │  Terminal │   │ Reading  │   │ Writing  │              │
│  │           │   │          │   │          │              │
│  └───────────┘   └──────────┘   └──────────┘              │
│                                                             │
│  ┌───────────┐   ┌──────────┐                              │
│  │    Web    │   │ Thinking │                              │
│  │           │   │          │                              │
│  └───────────┘   └──────────┘                              │
│                                                             │
└─────────────────────────┬───────────────────────────────────┘
                          │
              tool_end event
                          │
                          ▼
┌─────────────────────────────────────────────────────────────┐
│                    Speaking Layer                           │
│                                                             │
│  ┌────────────────┐                                         │
│  │ Turn to Camera │ ──► Talking ──► Turn from Camera       │
│  └────────────────┘                                         │
│                                                             │
└─────────────────────────────────────────────────────────────┘
```

### State Priorities

When multiple animation triggers occur:

| Priority | Layer | Condition |
|----------|-------|-----------|
| 1 (highest) | Speaking | Assistant is sending text |
| 2 | Work | Tool is executing |
| 3 (lowest) | Idle | No other activity |

**Blending**: Higher-priority layers override lower layers with smooth transitions (500ms blend time).

---

## Animation Controller

```rust
pub struct AnimationController {
    // Current state
    body_state: BodyState,
    face_state: FaceState,

    // Animation machine
    animation_machine: Handle<Machine>,

    // Timing
    idle_variation_timer: f32,
    idle_variation_interval: f32,  // 10-30 seconds

    // Tool tracking
    active_tool: Option<String>,
    speaking: bool,
}

#[derive(Debug, Clone, PartialEq)]
pub enum BodyState {
    Idle(IdleVariant),
    Working(WorkAnimation),
    TurningToCamera,
    TurningFromCamera,
    Talking,
}

#[derive(Debug, Clone, PartialEq)]
pub enum IdleVariant {
    Sitting,
    Thinking,
    Stretching,
    LookingAround,
    Wandering,  // Agent strolls to random positions in the room
}

#[derive(Debug, Clone, PartialEq)]
pub enum WorkAnimation {
    Terminal,      // Typing on keyboard
    Reading,       // Reading document
    Writing,       // Writing on paper
    WebBrowsing,   // Looking at screen, mouse movements
    DeepThinking,  // Leaning back, hand on chin
}

#[derive(Debug, Clone, PartialEq)]
pub enum FaceState {
    Neutral,
    Talking { intensity: f32 },  // 0.0 to 1.0
    Thinking,                      // Furrowed brow
    Pleased,                       // Slight smile
}

impl AnimationController {
    pub fn update(&mut self, dt: f32, scene: &mut Scene) {
        // Update idle variation timer
        if matches!(self.body_state, BodyState::Idle(_)) {
            self.idle_variation_timer += dt;
            if self.idle_variation_timer >= self.idle_variation_interval {
                self.trigger_idle_variation();
                self.idle_variation_timer = 0.0;
                self.idle_variation_interval = rand::thread_rng().gen_range(10.0..30.0);
            }
        }

        // Update animation machine
        scene.graph[self.animation_machine]
            .as_animation_machine_mut()
            .update(dt);
    }

    pub fn on_tool_start(&mut self, tool: &str, scene: &mut Scene) {
        self.active_tool = Some(tool.to_string());

        let work_anim = Self::categorize_tool(tool);
        self.body_state = BodyState::Working(work_anim);

        // Send transition event to animation machine
        self.send_event("tool_start", work_anim.as_str(), scene);
    }

    pub fn on_tool_end(&mut self, scene: &mut Scene) {
        self.active_tool = None;

        // Return to idle if not speaking
        if !self.speaking {
            self.body_state = BodyState::Idle(IdleVariant::Sitting);
            self.send_event("tool_end", "idle_sitting", scene);
        }
    }

    pub fn on_speaking_start(&mut self, scene: &mut Scene) {
        self.speaking = true;
        self.body_state = BodyState::TurningToCamera;
        self.send_event("speaking_start", "turn_to_camera", scene);

        // After turn animation (1s), start talking
        // TODO: Use animation event callback when turn_to_camera finishes
    }

    pub fn on_speaking_end(&mut self, scene: &mut Scene) {
        self.speaking = false;
        self.body_state = BodyState::TurningFromCamera;
        self.send_event("speaking_end", "turn_from_camera", scene);

        // After turn animation (1s), return to idle or tool animation
        // TODO: Use animation event callback
    }

    pub fn on_text_delta(&mut self, _content: &str, scene: &mut Scene) {
        // Update talking animation intensity based on text rate
        // For now, just ensure we're in talking state
        if matches!(self.body_state, BodyState::TurningToCamera) {
            // Wait for turn to finish
        } else if self.speaking {
            self.body_state = BodyState::Talking;
            self.send_event("talking", "talking", scene);
        }
    }

    fn trigger_idle_variation(&mut self) {
        // Randomly pick idle variant (20% chance for each)
        use rand::Rng;
        let mut rng = rand::thread_rng();
        let variant = match rng.gen_range(0..5) {
            0 => IdleVariant::Sitting,
            1 => IdleVariant::Thinking,
            2 => IdleVariant::Stretching,
            3 => IdleVariant::LookingAround,
            _ => IdleVariant::Wandering,
        };

        self.body_state = BodyState::Idle(variant);
        // TODO: Send event to animation machine
    }

    fn categorize_tool(name: &str) -> WorkAnimation {
        match name {
            "Bash" | "BashOutput" | "Execute" => WorkAnimation::Terminal,
            "Read" | "Glob" | "Grep" | "View" => WorkAnimation::Reading,
            "Write" | "Edit" | "MultiEdit" => WorkAnimation::Writing,
            "WebSearch" | "WebFetch" | "web_search" | "web_fetch" => WorkAnimation::WebBrowsing,
            "Task" | "Explore" => WorkAnimation::DeepThinking,
            "mcp__room-tools__create_note" => WorkAnimation::Writing,
            "mcp__room-tools__look_around" | "mcp__room-tools__examine" => WorkAnimation::Reading,
            _ => WorkAnimation::DeepThinking,  // Fallback
        }
    }

    fn send_event(&self, event_name: &str, target_animation: &str, scene: &mut Scene) {
        // TODO: Fyrox animation machine API for events
        // This would trigger a transition in the animation state machine
    }
}
```

---

## Fyrox Animation Machine Setup

Fyrox provides a visual animation editor for creating state machines. Our machine would have:

### States

Each state plays an animation clip (looping or one-shot):

```rust
State::new("idle_sitting")
    .with_animation("idle_sitting")
    .with_loop(true)

State::new("turn_to_camera")
    .with_animation("turn_to_camera")
    .with_loop(false)
    .with_exit_event("turn_complete")  // Trigger when animation finishes

State::new("talking")
    .with_animation("talking")
    .with_loop(true)
```

### Transitions

Transitions define when to move between states:

```rust
Transition::new("idle_sitting", "turn_to_camera")
    .with_condition(Condition::Event("speaking_start"))
    .with_blend_time(0.3)

Transition::new("turn_to_camera", "talking")
    .with_condition(Condition::Event("turn_complete"))
    .with_blend_time(0.2)

Transition::new("talking", "turn_from_camera")
    .with_condition(Condition::Event("speaking_end"))
    .with_blend_time(0.3)

Transition::new("turn_from_camera", "idle_sitting")
    .with_condition(Condition::Event("turn_complete"))
    .with_blend_time(0.5)
```

**Note**: This is pseudocode. Actual Fyrox API uses `MachineBuilder` or visual editor.

---

## Face Animation

### Blend Shapes (Morph Targets)

Face expressions use blend shapes (vertex deformation):

| Blend Shape | Purpose |
|-------------|---------|
| `jaw_open` | Talking (opens mouth) |
| `brow_down` | Thinking (furrowed brow) |
| `smile` | Pleased expression |
| `blink` | Eye blink (automatic, random) |

### Face Controller

```rust
pub struct FaceController {
    state: FaceState,
    mesh: Handle<Mesh>,

    // Blend shape weights (0.0 to 1.0)
    jaw_open: f32,
    brow_down: f32,
    smile: f32,

    // Blink timer
    blink_timer: f32,
    next_blink: f32,
}

impl FaceController {
    pub fn update(&mut self, dt: f32, scene: &mut Scene) {
        // Smooth blend shape interpolation
        match self.state {
            FaceState::Neutral => {
                self.jaw_open = Self::lerp(self.jaw_open, 0.0, dt * 5.0);
                self.brow_down = Self::lerp(self.brow_down, 0.0, dt * 5.0);
                self.smile = Self::lerp(self.smile, 0.0, dt * 5.0);
            }
            FaceState::Talking { intensity } => {
                // Animate jaw opening (simple noise function for now)
                let target = (self.jaw_open + dt * 10.0).sin().abs() * intensity;
                self.jaw_open = Self::lerp(self.jaw_open, target, dt * 20.0);
            }
            FaceState::Thinking => {
                self.brow_down = Self::lerp(self.brow_down, 0.6, dt * 3.0);
            }
            FaceState::Pleased => {
                self.smile = Self::lerp(self.smile, 0.5, dt * 3.0);
            }
        }

        // Automatic blinking
        self.blink_timer += dt;
        if self.blink_timer >= self.next_blink {
            self.trigger_blink();
            self.blink_timer = 0.0;
            self.next_blink = rand::thread_rng().gen_range(2.0..6.0);
        }

        // Apply blend shapes to mesh
        if let Some(mesh) = scene.graph.try_get_mut(self.mesh) {
            if let Some(skinned) = mesh.as_mesh_mut() {
                skinned.set_blend_shape_weight("jaw_open", self.jaw_open);
                skinned.set_blend_shape_weight("brow_down", self.brow_down);
                skinned.set_blend_shape_weight("smile", self.smile);
            }
        }
    }

    fn lerp(current: f32, target: f32, alpha: f32) -> f32 {
        current + (target - current) * alpha.min(1.0)
    }

    fn trigger_blink(&self) {
        // TODO: Play blink animation (quick close-open)
    }
}
```

**Rationale**:
- **Blend shapes over skeletal**: Faces need subtle deformations that bones can't provide
- **Simple talking**: Jaw movement based on sine wave (cheap approximation of speech)
- **Automatic blink**: Adds life without backend coordination

---

## Camera Control

### Current Implementation (MVP)

The camera uses a **fixed isometric view** from the "open corner" of the room:

| Property | Value | Notes |
|----------|-------|-------|
| **Position** | `(7.0, 7.0, 7.0)` | Elevated, equal X/Z for true isometric |
| **Look Target** | `(-0.5, 0.0, -0.5)` | Toward room center, offset to wall corner |
| **Projection** | Orthographic | No perspective distortion |
| **Vertical Size** | `6.0` | Controls zoom (smaller = more zoomed in) |

This creates a classic isometric RPG-style view looking diagonally into the Victorian study from the front-right corner. The room is 6×6 units with walls at X=-3 (left) and Z=-3 (back), forming a cozy corner.

**Implementation** (in `game/src/lib.rs`, `on_scene_loaded`):
- Camera position and rotation set programmatically
- Orthographic projection applied via `camera.set_projection()`
- Fallback warnings logged if camera node not found or not a Camera component

### Future Enhancement: Dynamic Camera Focus

The following describes planned dynamic camera behavior (not yet implemented):

```rust
pub struct CameraController {
    camera: Handle<Node>,
    // Fixed isometric position (current MVP default)
    default_position: Vec3,  // (7.0, 7.0, 7.0)
    default_target: Vec3,    // (-0.5, 0.0, -0.5)

    // Focus state (future enhancement)
    focusing_on: Option<FocusTarget>,
    focus_progress: f32,  // 0.0 to 1.0
}

#[derive(Debug, Clone, PartialEq)]
pub enum FocusTarget {
    Character,    // During speaking (future)
    Subagent(String),  // When subagent spawns (future)
}

impl CameraController {
    // FUTURE: Dynamic camera control - not implemented in MVP
    pub fn focus_on_character(&mut self) {
        self.focusing_on = Some(FocusTarget::Character);
        self.focus_progress = 0.0;
    }

    pub fn update(&mut self, dt: f32, scene: &mut Scene) {
        // FUTURE: Implement smooth camera transitions
        // For MVP, camera remains fixed at isometric position

        let target_pos = match &self.focusing_on {
            Some(FocusTarget::Character) => {
                // Move camera closer to character's face
                Vec3::new(0.0, 1.5, 2.0)
            }
            Some(FocusTarget::Subagent(_id)) => {
                // Look at subagent position
                Vec3::new(2.0, 1.5, 2.0)
            }
            None => self.default_position,
        };

        // Smooth interpolation (2-second transition)
        self.focus_progress = (self.focus_progress + dt * 0.5).min(1.0);
        // ... lerp camera position ...
    }
}
```

**Rationale (Current MVP)**:
- **Fixed isometric view**: Consistent environmental visibility, classic aesthetic
- **Orthographic projection**: True isometric proportions without perspective distortion
- **Simple implementation**: No complex camera state management needed yet

**Future (Phase 2)**:
- **Focus on character**: When speaking, camera smoothly pans closer (better view of expressions)
- **Focus on subagent**: When subagent spawns, camera shows spatial relationship
- **Smooth transitions**: 2-second interpolation avoids jarring jumps

---

## Subagent Visualization

**Open design question**: How do we represent subagents in the 3D scene?

### Option 1: Separate Character

- Spawn a second character model at the "lectern" surface
- Same rigging, different materials/colors (e.g., blue tint for subagent)
- When subagent is working, animate it independently

**Pros**: Clear visual distinction, can show parallel work
**Cons**: Complexity (managing two animation controllers), memory overhead

### Option 2: Particle Effect

- Spawn glowing orb or particle effect at lectern
- Pulsates when subagent is active
- No detailed animation needed

**Pros**: Simple, low overhead
**Cons**: Less immersive, no character presence

### Option 3: UI Indicator Only

- No 3D representation
- Show subagent status in UI overlay (e.g., "Subagent: Searching codebase...")

**Pros**: Simplest
**Cons**: Breaks immersion, not spatial

**Recommendation**: Start with Option 3 (UI only) for MVP, consider Option 1 for future polish.

---

## Performance

### Animation Playback

- **Skeletal animation**: ~0.5ms per character at 50 bones
- **Blend shape updates**: ~0.1ms per frame (4 blend shapes)
- **State machine updates**: <0.1ms

**Total**: <1ms per frame (well under 16ms budget for 60fps)

### Memory

- **Animation clips**: ~6MB (all clips loaded)
- **Character mesh**: ~500KB (skinned mesh + textures)
- **State machine**: ~10KB

**Total**: ~7MB

---

## Sound System (MVP)

The sound system provides audio feedback for agent interactions, implemented in `game/src/audio/mod.rs`.

### Sound Style Philosophy

**Hybrid Aesthetic:** The agent "speaks" in chiptune (digital entity) but interacts with the Victorian study's physical objects using realistic analog sounds. Digital being in an analog world!

### Implemented Sounds

| Event | Sound | Implementation |
|-------|-------|----------------|
| Assistant speaking | Chiptune beep (looped) | `beep_mumble.wav` loops at ~150ms interval during `AssistantTextDelta` |
| Open note/image | Paper rustle | `paper_rustle.wav` plays once when viewer opens |

### SoundManager

```rust
pub struct SoundManager {
    mumble_buffer: Option<SoundBufferResource>,
    paper_rustle_buffer: Option<SoundBufferResource>,
    is_speaking: bool,
    last_beep_time: Option<Instant>,
    master_volume: f32,
    audio_enabled: bool,
    active_sounds: Vec<Handle<Node>>,
}
```

**Key methods:**
- `start_speaking()` / `stop_speaking()` - Control mumble loop state
- `update(scene)` - Called each frame, plays beeps at interval during speaking
- `play_paper_rustle(scene)` - One-shot paper sound
- `set_volume(volume, scene)` / `set_enabled(enabled, scene)` - Settings integration

### Usage Flow

1. **Initialization**: `SoundManager::new(volume, enabled)` in `Game::init()`
2. **Asset loading**: `load_sounds(resource_manager)` in `on_scene_loaded()`
3. **Speaking**: `start_speaking()` on first `AssistantTextDelta`, `stop_speaking()` on `AssistantMessageComplete`
4. **Frame update**: `update(scene)` handles timed beep playback
5. **Settings**: `apply_audio_settings()` syncs Settings dialog changes

### Future Expansion (Deferred)

Tool category sounds (terminal typing, page flips) are intentionally omitted for MVP to avoid audio fatigue. The infrastructure supports adding more sounds later by:
- Adding new `SoundBufferResource` fields
- Creating play methods following `play_paper_rustle` pattern
- Wiring to appropriate events in `lib.rs`

---

## Open Questions

1. ~~**Audio sync**: Should we play audio (keyboard typing, paper rustling) with animations?~~
   - **RESOLVED**: Implemented in `game/src/audio/mod.rs`
   - Mumble beeps loop during assistant speaking (Animal Crossing style)
   - Paper rustle plays when opening notes/images
   - Volume and enable/disable controllable via Settings dialog
   - Kept minimal to avoid audio fatigue (no tool sounds for MVP)

2. **Animation events**: How do we know when "turn_to_camera" finishes?
   - Fyrox supports animation event markers
   - Need to author events in Blender and export
   - Alternative: Fixed timer (less accurate, simpler)

3. **Subagent animation**: See "Subagent Visualization" above

4. **Gesture timing**: When should character gesture while talking?
   - Random (every 5-10 seconds)?
   - Based on text content (punctuation, emphasis)?
   - Manual trigger from backend (agent sends "gesture" hint)?

5. **Tool animation feedback**: Should tools send progress updates?
   - E.g., "Bash: still running... (5s)"
   - Would allow different animations for quick vs slow tools
   - Current: Single animation regardless of duration

---

## Related Documents

- [Frontend Architecture](./frontend-architecture.md) - Animation controller integration
- [IPC Protocol](./ipc-protocol.md) - Tool events that trigger animations
- [Backend Adapter](./backend-adapter.md) - Subagent message handling