A 3D interactive AI agent application featuring a Victorian study environment where an AI character works alongside you. Built with Fyrox (Rust) and the Claude Agent SDK.

Overview #

Tiny Workshop creates an immersive, cozy workspace where you can interact with an AI agent character in a fully-rendered 3D Victorian study. The agent can create notes, organize images, and manipulate objects in the room while conversing with you through a natural chat interface.

Key Features #

Implemented:

Chat Interface: Fyrox-UI chat interface with streaming message support
z.AI API Key Authentication: Simple API key authentication via OS keychain
Cross-Platform: Works on Windows, macOS, and Linux
IPC Communication: Robust JSON-RPC protocol between frontend and backend
Error Recovery: Auto-restart on backend crash (up to 3 attempts with 1-second delays)
Debug Server: HTTP API for automated testing (feature-gated)
Room Persistence: Room state persists across sessions with atomic writes and multi-instance locking
Session Logs: JSONL-based conversation history with crash-safe append-only semantics
Session Resume: Conversations persist across app restarts with full SDK context preservation
Image Texture Rendering: Full texture support for room images using MaterialResource::new_embedded() with LRU cache
Text Rendering: Sticky notes display rendered text (title and content preview) using raqote and font_kit
Settings Dialog: Configurable application settings (Graphics quality, Audio volume, API key management, Advanced info) with persistent storage and real-time engine application

Planned:

3D Victorian Study Environment: Beautiful, cozy workspace rendered in real-time (currently using placeholder geometry)
Intelligent Character: AI agent with natural animations and interactions (placeholder cube for now)
Optimized Performance: Runs smoothly on integrated graphics (<200MB RAM, 60fps target)

Architecture #

Tiny Workshop uses a multi-process architecture for separation of concerns:

┌─────────────────┐         IPC (JSON-RPC)        ┌─────────────────┐
│  Fyrox Frontend │ ◄──────────────────────────► │ Backend Adapter │
│     (Rust)      │    stdin/stdout messages      │ (Bun/TypeScript)│
└─────────────────┘                               └────────┬────────┘
        │                                                  │
        ├─ 3D Rendering (60fps)                          │
        ├─ Animation System                               │
        ├─ Room Management                                ▼
        ├─ Session Persistence               ┌────────────────────┐
        └─ Chat UI (Fyrox-UI)                │ Claude Agent SDK   │
                                              └────────────────────┘

Components #

Frontend (Rust/Fyrox): Handles 3D rendering, animations, room state, UI, and persistence
Backend Adapter (Bun/TypeScript): Bridges the frontend to the Claude Agent SDK via IPC
MCP Server: In-process room tools server that allows the agent to interact with the environment

Project Structure #

tiny-workshop/
├── docs/
│   └── design/              # Architecture and design documents
├── tiny-workshop/           # Main Rust workspace
│   ├── editor/             # Fyrox editor for scene authoring
│   ├── executor/           # Desktop runtime (main application)
│   ├── executor-wasm/      # WebAssembly runtime (future)
│   ├── executor-android/   # Android runtime (future)
│   ├── game/               # Core game logic (static linking)
│   │   └── src/
│   │       ├── lib.rs              # Main Game plugin
│   │       ├── ipc.rs              # IPC protocol types
│   │       ├── backend.rs          # Backend process management
│   │       ├── chat_ui.rs          # Chat interface
│   │       ├── credential_manager.rs # OS keychain integration (with Linux fallback)
│   │       ├── debug_server/       # HTTP debug API (feature-gated)
│   │       ├── persistence/        # Settings and state persistence
│   │       ├── settings_dialog.rs  # Settings UI modal dialog
│   │       └── room/               # Room system module
│   │           ├── mod.rs          # Surface/interactable management
│   │           ├── fonts.rs        # Font loading for text rendering
│   │           ├── text_renderer.rs # Text-to-PNG rendering
│   │           ├── image_cache.rs  # LRU texture cache
│   │           └── image_storage.rs # Image file management
│   └── game-dylib/         # Hot-reloadable game plugin (development)
├── backend/                # TypeScript backend adapter
│   └── packages/
│       ├── ipc-types/      # IPC protocol definitions
│       ├── logger/         # Structured logging
│       └── adapter/        # Main adapter (SDK integration)
├── CONTRIBUTING.md         # Contribution guidelines
└── README.md              # This file

Development Status #

Current Phase: Active Development (Phase 4 - Character & Animation)

Core systems are complete! IPC communication, authentication, chat UI, animation state system, full room system (with text rendering, image stacks, and all 5 surfaces), and persistence are all working. Next up: 3D scene and character models.

What's Implemented #

Frontend (Rust)

Main Game plugin with backend IPC integration
IPC protocol types and backend process management
Backend crash recovery with auto-restart (up to 3 attempts)
Fyrox-UI chat interface with streaming message support and system status messages
OS keychain integration (Windows Credential Manager, macOS Keychain, Linux Secret Service with config file fallback)
z.AI API key authentication
Animation state system (placeholder cube with color-coded states for idle, working, speaking)
HTTP debug server for automated testing (feature-gated)
Room state persistence with atomic writes, schema versioning, and multi-instance file locking
Session logging with JSONL format, append-only semantics for crash safety
Room system with surfaces and interactables (sticky notes, image stacks)
Text rendering for sticky notes using raqote (renders to PNG textures)
Image texture rendering with LRU cache (50 images or 50MB limit)

Backend (TypeScript)

IPC protocol type definitions
z.AI API key authentication
Main adapter with multi-turn SDK integration (InputGenerator pattern)

Not Yet Implemented #

3D scene and character models (placeholder cube in place for animation testing)

Design Documents #

Comprehensive design documents are available in docs/design/:

Frontend Architecture - Fyrox-based 3D application
Backend Adapter - TypeScript/Bun IPC bridge
IPC Protocol - Communication protocol
Room System - Interactive environment management
Animation System - Character animations
Persistence System - State management
Authentication System - z.AI API key authentication
Debug Server - HTTP debug API for automated testing
Open Issues - Known gaps and design questions

Quick Start #

Prerequisites #

Rust: Install via rustup
Bun: Install via bun.sh
Platform-specific dependencies (see CONTRIBUTING.md)

Building #

cd tiny-workshop
cargo build --release

Running #

cargo run --package executor --release

Debug Server (Optional) #

For automated testing, you can enable an HTTP debug server:

./dev.sh --debug-server      # Linux/macOS
dev.bat --debug-server       # Windows

This exposes endpoints at http://localhost:8765 for sending messages and querying state. See CONTRIBUTING.md for details.

Keyboard Shortcuts #

Shortcut	Action
`Ctrl+,`	Open Settings dialog
`Escape`	Close Settings dialog (when open)
`Enter`	Send chat message
`Shift+Enter`	Insert newline in chat

Testing #

# Run all Rust tests
cd tiny-workshop && cargo test

# Run all backend tests
cd backend && pnpm test

# Run integration tests (requires backend build)
cd backend && pnpm build
cd ../tiny-workshop && cargo test --test ipc_integration

For detailed testing instructions, see CONTRIBUTING.md.

For detailed development instructions, see CONTRIBUTING.md.

Room System (Planned) #

Once implemented, the agent will interact with the room environment through natural language commands. The Victorian study will include:

Desk: Main workspace for images and tools (6 slots)
Windowside Table: For notes and reference materials (4 slots)
Mantelpiece: For important items or decorative objects (3 slots)
Bookshelf: For reference materials and organization (8 slots)
Lectern: Standing workspace for subagents (2 slots)

The agent will be able to:

Create and organize sticky notes with custom text and colors
Place and stack images (screenshots, diagrams, etc.)
Query the current room state to remember what's been placed
Organize items spatially for better workflow

Performance Targets #

Frame Rate: 60fps on integrated graphics
Memory: <200MB RAM total
Startup Time: <2 seconds
Platform: Windows, macOS, Linux

Technology Stack #

Component	Technology	Purpose
Frontend	Fyrox 1.0.0-rc.1	3D game engine (Rust)
UI	Fyrox-UI	Retained-mode UI overlay
Backend	Bun + TypeScript	SDK adapter and IPC bridge
SDK	Claude Agent SDK	AI agent capabilities
IPC	JSON-RPC over stdin/stdout	Process communication
Persistence	serde + JSON	State serialization
Auth	keyring	Secure credential storage

Contributing #

Contributions are welcome! Please read CONTRIBUTING.md for:

Development setup instructions
Code style guidelines
Testing procedures
Contribution workflow
Design document references

Design Philosophy #

Tiny Workshop prioritizes:

Coziness: A warm, inviting environment that feels lived-in
Performance: Smooth 60fps on modest hardware
Persistence: Everything you create stays with you across sessions
Simplicity: Clean architecture with clear separation of concerns
Reliability: Crash recovery, session resumption, and error handling

License #

See LICENSE file for details.

Acknowledgments #

Built with Fyrox game engine
Powered by Claude Agent SDK
Runtime by Bun

Clone this repository

Tiny Workshop #