karashiiro.moe / tiny-workshop
WIP experiment with 95% vibe coding
Star
Fork 0
Watch
tiny-workshop docs design debug-server.md
13 kB
Markdown
at main

Debug Server #

An optional HTTP REST API debug server that enables automated testing and external tooling to interact with the Tiny Workshop frontend without using the UI.

Overview #

The debug server runs as a separate thread within the frontend process, exposing HTTP endpoints for:

  • Sending user messages (triggers full end-to-end flow with backend)
  • Querying game state (authentication status, session info, backend connection)
  • Retrieving conversation history (for polling async responses)

This enables Claude Code (or other tools) to drive the application programmatically for testing purposes.

Enabling the Debug Server #

The debug server is feature-gated and disabled by default.

Build with Debug Server #

cargo build --package executor --features debug_server

Run with Debug Server #

# Default port (8765)
cargo run --package executor --features debug_server

# Custom port via environment variable
TINY_WORKSHOP_DEBUG_PORT=9000 cargo run --package executor --features debug_server

Port Configuration #

Priority order:

  1. TINY_WORKSHOP_DEBUG_PORT environment variable
  2. Default: 8765

API Endpoints #

GET /health #

Health check endpoint.

Response 200 OK:

{
  "status": "ok",
  "version": "0.1.0"
}

GET /state #

Query current game state.

Response 200 OK:

{
  "session_id": "abc-123-def",
  "is_authenticated": true,
  "backend_connected": true,
  "backend_disconnected": false,
  "waiting_for_auth_code": false,
  "current_response_length": 0,
  "has_chat_ui": true
}

POST /send #

Send a user message through the full end-to-end flow.

Request:

{
  "content": "Hello, Claude!"
}

Response 200 OK:

{
  "success": true,
  "message_id": 1
}

Error Response 500 Internal Server Error:

{
  "success": false,
  "error": "Backend not connected"
}

GET /messages #

Get recent conversation messages. Use this to poll for responses after sending a message.

Query Parameters:

  • limit (optional): Maximum messages to return (default: 50, max: 100)
  • after (optional): Only return messages with ID greater than this value

Response 200 OK:

{
  "messages": [
    {
      "id": 1,
      "role": "user",
      "content": "Hello!",
      "timestamp": 1704326400000
    },
    {
      "id": 2,
      "role": "assistant",
      "content": "Hi there! How can I help you today?",
      "timestamp": 1704326401500
    }
  ],
  "has_more": false
}

Message Roles:

  • user - Messages sent by the user
  • assistant - Responses from the AI assistant
  • tool - Tool execution indicators
  • system - System messages (auth prompts, errors, etc.)

GET /screenshot #

Capture the current window as a JPEG image, including both the 3D scene and UI elements.

Response 200 OK:

  • Content-Type: image/jpeg
  • Body: JPEG image data

Error Response 500 Internal Server Error:

  • Plain text error message

Error Response 504 Gateway Timeout:

  • Plain text: "Screenshot capture timed out"

Usage:

# Save screenshot to file
curl -o screenshot.jpg http://localhost:8765/screenshot

# Or open directly in browser
open http://localhost:8765/screenshot

Notes:

  • Screenshots include the complete rendered frame (3D scene + UI)
  • Screenshot capture uses async GPU readback and may take 1-2 frames to complete
  • Timeout: 10 seconds (screenshots span multiple frames)
  • Returns RGBA pixels encoded as JPEG (quality 85)
  • Images are vertically flipped from GPU to match standard image orientation

GET /session-picker #

Query the session picker state.

Response 200 OK:

{
  "visible": true,
  "sessions": [
    {
      "index": 0,
      "session_id": "abc-123",
      "name": "Session 1",
      "created_at": "2025-01-12T10:00:00Z",
      "updated_at": "2025-01-12T11:30:00Z"
    }
  ],
  "selected_index": 0
}

POST /session-picker/action #

Perform an action on the session picker.

Request (select session by index):

{
  "type": "select_session",
  "index": 0
}

Request (resume selected session):

{
  "type": "resume_selected"
}

Request (start new session):

{
  "type": "new_session"
}

Response 200 OK:

{
  "success": true
}

POST /click #

Simulate a mouse click at screen coordinates. Used for testing 3D interactions like clicking on sticky notes.

Request:

{
  "x": 400.0,
  "y": 300.0
}

Response 200 OK:

{
  "hit": true,
  "hit_note_id": "uuid-of-note",
  "hit_note_title": "Note Title"
}

If nothing was hit:

{
  "hit": false
}

Notes:

  • Coordinates are screen pixels from top-left origin
  • Currently only detects sticky note meshes
  • If a note is hit, opens the note viewer modal

POST /ui/visibility #

Show or hide UI elements for testing. Useful for getting a clear view of the 3D scene.

Request:

{
  "chat": false
}

Response 200 OK:

{
  "success": true,
  "chat_visible": false
}

Notes:

  • The chat UI covers most of the screen by default
  • Hide it to see the full 3D scene for visual inspection
  • Hide it to click on notes that would be covered by the chat window

GET /settings #

Query the settings dialog state.

Response 200 OK:

{
  "visible": false,
  "active_tab": 0,
  "settings": {
    "quality_preset": "Medium",
    "vsync_enabled": true,
    "fps_limit": "Fps60",
    "audio_enabled": true,
    "volume": 0.8,
    "update_channel": "Stable"
  }
}

Tab Indices: 0 = Graphics, 1 = Audio, 2 = Auth, 3 = Advanced

POST /settings/action #

Perform an action on the settings dialog.

Open settings:

{"type": "open"}

Close settings (cancel):

{"type": "close"}

Save settings and close:

{"type": "save"}

Switch to a tab (0-3):

{"type": "switch_tab", "index": 1}

Set graphics quality (0=Low, 1=Medium, 2=High):

{"type": "set_quality", "index": 1}

Set VSync:

{"type": "set_vsync", "enabled": true}

Set FPS limit (0=Unlimited, 1=30, 2=60, 3=120, 4=144):

{"type": "set_fps_limit", "index": 2}

Set audio enabled:

{"type": "set_audio_enabled", "enabled": true}

Set volume (0.0 to 1.0):

{"type": "set_volume", "volume": 0.75}

Set API key:

{"type": "set_api_key", "key": "sk-..."}

Simulate Escape key (tests pending_close_settings code path):

{"type": "simulate_escape_key"}

Response 200 OK:

{"success": true}

Error Response:

{"success": false, "error": "Settings dialog not open"}

Notes:

  • Settings dialog can also be opened with Ctrl+, keyboard shortcut
  • Escape key closes the dialog (simulated via simulate_escape_key)
  • Changes are not persisted until Save is clicked
  • API key updates are stored immediately via CredentialManager

Polling Workflow #

Since AI responses are asynchronous, you need to poll for them:

# 1. Check health
curl http://localhost:8765/health

# 2. Verify authenticated (check is_authenticated in response)
curl http://localhost:8765/state

# 3. Send a message (note the message_id returned)
curl -X POST http://localhost:8765/send \
  -H "Content-Type: application/json" \
  -d '{"content": "What is 2+2?"}'

# 4. Poll for response (repeat until assistant message appears)
curl "http://localhost:8765/messages?after=1"

# 5. Look for role: "assistant" in the response

Polling Strategy #

┌─────────────────┐
│  POST /send     │
│  → message_id   │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│ GET /messages   │◄────┐
│ ?after={id}     │     │
└────────┬────────┘     │
         │              │
         ▼              │
    Has assistant       │
    message? ───No──────┘
         │
        Yes
         │
         ▼
    ┌─────────┐
    │  Done!  │
    └─────────┘

Recommended polling interval: 100-500ms

Architecture #

The debug server follows the same threading pattern as the backend IPC:

Game Loop Thread                 Debug Server Thread
      │                                 │
      │                          ┌──────┴──────┐
      │                          │ axum router │
      │                          │ (port 8765) │
      │                          └──────┬──────┘
      │                                 │
      │                                 ▼
      │◄─── mpsc::Receiver ───── command_tx.send()
      │     (DebugCommand)              │
      │                                 │
      │                          ┌──────┴──────┐
      │                          │   oneshot   │
      │                          │   channel   │
      │                          └──────┬──────┘
      │                                 │
      └── handle.send_response() ──────►│
          (DebugResponse)               │
                                        ▼
                                  HTTP response

Key Design Points #

  1. Separate Thread: The HTTP server runs on its own thread with a dedicated tokio runtime, keeping the game loop unblocked.

  2. Non-blocking Communication: Uses mpsc channels for commands and oneshot channels for responses.

  3. Rate Limited: The game loop processes max 10 debug commands per frame (same as backend messages) to maintain 60fps.

  4. Request Correlation: Each request gets a unique ID to match responses back to waiting HTTP handlers.

  5. Message History: Stores up to 200 messages in a ring buffer for the /messages endpoint.

Security Considerations #

  1. Localhost Only: The server binds to 127.0.0.1, not 0.0.0.0. Only local processes can connect.

  2. No Auth Bypass: Sending messages through the debug server still requires valid authentication. The messages flow through the normal backend which validates credentials.

  3. Feature Gated: The debug server code is not included in release builds unless --features debug_server is explicitly specified.

  4. No Sensitive Data Exposure: The /state endpoint does not expose tokens or credentials.

Message History #

The debug server maintains a bounded message history for the /messages endpoint:

  • Capacity: 200 messages maximum
  • Eviction: When full, oldest messages are evicted (FIFO)
  • Persistence: Not persisted - cleared on restart
  • Tracking: Messages from both user input and assistant responses are tracked

Messages are recorded when:

  • User sends a message (via UI or debug server)
  • Assistant response delta is received (accumulated into complete message)
  • Tool execution starts/ends
  • System events occur (auth prompts, errors)

Example Usage #

Basic Health Check #

curl http://localhost:8765/health
# {"status":"ok","version":"0.1.0"}

Check Authentication Status #

curl http://localhost:8765/state | jq '.is_authenticated'
# true

Send Message and Wait for Response #

#!/bin/bash

# Send message
response=$(curl -s -X POST http://localhost:8765/send \
  -H "Content-Type: application/json" \
  -d '{"content": "What is the capital of France?"}')

message_id=$(echo $response | jq '.message_id')

# Poll for response
while true; do
  messages=$(curl -s "http://localhost:8765/messages?after=$message_id")

  # Check if we have an assistant message
  has_response=$(echo $messages | jq '.messages | map(select(.role == "assistant")) | length > 0')

  if [ "$has_response" = "true" ]; then
    echo "Response received:"
    echo $messages | jq '.messages[] | select(.role == "assistant") | .content'
    break
  fi

  sleep 0.2
done

Troubleshooting #

"Connection refused" #

  • Ensure the debug server feature is enabled: --features debug_server
  • Check if another process is using the port
  • Verify the port with TINY_WORKSHOP_DEBUG_PORT env var

"Backend not connected" error #

  • The backend process may have crashed or failed to start
  • Check tiny-workshop.log for backend errors

"Not authenticated" in state #

  • Complete the OAuth flow through the UI first
  • Or check if the refresh token is expired

Messages not appearing #

  • Ensure you're polling with the correct after parameter
  • Check that the backend is actually connected
  • Verify authentication status