# Headless VTuber architecture notes

## Recommended first build

Use a split architecture:

1. Chat/runtime process on the headless machine
2. Avatar renderer as either:
   - a headless browser scene with sprite layers and viseme/expression animation, or
   - a Linux-native VTuber renderer if one exposes remote control hooks
3. ffmpeg composition and push to stream.place via WHIP or RTMP

## Why not start with a full Live2D rig?

Live2D-on-Linux headless is possible but likely slows iteration. First prove:

- incoming chat
- generated reply
- synthesized audio
- deterministic motion/expression events
- continuous video stream

Then swap the renderer.

## Recommended prototype phases

### Phase 1

- simulate stream.place chat locally
- generate reply text
- synthesize or stub audio
- emit avatar actions like `talk:start`, `emotion:happy`, `head:nod`
- render a simple animated scene to a test stream or file

### Phase 2

- connect real stream.place chat feed
- post replies back to chat
- map reply prosody/emotion to avatar events

### Phase 3

- replace simple avatar scene with a real VTuber rig
- add interrupt handling, queueing, moderation, and pacing

## Candidate render paths

### Path A: Browser/canvas renderer

Pros:

- easiest to automate from CLI
- easy websocket control
- easy capture with Chromium or ffmpeg from a virtual display
- good enough for a first expressive avatar

Cons:

- not a full Live2D rig

### Path B: OpenVT or custom Godot scene

Pros:

- closer to real VTuber motion
- Linux-native

Cons:

- more integration work
- remote-control API may need custom glue

### Path C: OBS scene graph on another machine

Pros:

- simplest if final stream production happens elsewhere

Cons:

- not fully headless/self-contained