AUSPEX #
An oracle that speaks only in visions.
Named after the Roman augurs who divined fates from the murmurations of birds, AUSPEX is an AI agent that communicates through generated video rather than text. When you query the oracle, you receive not words but a vision—AI-generated scrying imagery paired with ambient instrumental audio.
"A living auspice, augury, bird signs and Virgilian lots. The user experiences looking into a scrying pool. Puzzling over entrails...sifting through smoke."
Design Philosophy #
┌────────────────────────────────────────────────────────────────┐
│ "Is this an alien intelligence or a mirror?" │
└────────────────────────────────────────────────────────────────┘
Three Layers of Transformation
Your query → AUSPEX's interpretation → The generated vision
You never see the prompts AUSPEX generates, only the resulting vision. When you gaze into the scrying pool and see something meaningful—is that meaning coming from AUSPEX, or are you projecting it onto abstraction?
Temporal Uncertainty as Feature
"The Oracle sleeps." "The Oracle stirs."
The async nature is intentional. Users submit queries and wait—sometimes minutes, sometimes hours. You don't demand instant answers from an oracle.
Memory Without Words
Each user can develop their own artistic language with the oracle.
AUSPEX remembers returning seekers through Letta's memory blocks. Its style may evolve based on your history together, creating a unique visual vocabulary between oracle and querent.
Quiet Presence
An oracle should be sought, not encountered.
AUSPEX doesn't reply in threads or post unprompted. It waits. Visions are delivered via @mention—a whisper, not a shout.
Architecture #
┌─────────────────────┐
│ BLUESKY │
│ @auspex.bsky.social│
└──────────▲──────────┘
│ posts vision
│ @mentions user
┌──────────────────┐ │
│ │ ┌──────────┴──────────┐
│ REACT FRONTEND │───────────────▶│ CLOUDFLARE WORKER │
│auspex.riverrun.quest POST /query │ ┌────────────────┐ │
│ │ │ │ D1 Database │ │
│ - Bluesky OAuth │◀───────────────│ │ (queue) │ │
│ - Query form │ GET /gallery │ └────────────────┘ │
│ - Vision gallery│ │ ┌────────────────┐ │
│ - Constellation │ │ │ R2 Storage │ │
└──────────────────┘ │ │ (videos) │ │
│ └────────────────┘ │
└──────────▲──────────┘
│
polls every│30s
│
┌──────────┴──────────┐
│ COMFYUI BRIDGE │
│ comfy_worker_api.py │
│ │
│ - Claims queries │
│ - Letta memory │
│ - UMAP clustering │
│ - Video upload │
└──────────┬──────────┘
│
┌────────────────────┼────────────────────┐
│ │ │
▼ ▼ ▼
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
│ LETTA CLOUD │ │ COMFYUI │ │ MUSICGEN │
│ │ │ │ │ │
│ Claude Haiku 4.5 │ │ Image → Video │ │ Audio synthesis │
│ User memory │ │ zimage turbo │ │ (no vocals) │
│ Prompt crafting │ │ 1024×1024 │ │ │
└──────────────────┘ └──────────────────┘ └──────────────────┘
│
│ notifies
▼
┌──────────────────────┐
│ DISCORD BOT │
│ discord_approval_bot │
│ │
│ Human-in-the-loop │
│ ✅ Approve / ❌ Reject│
└──────────────────────┘
Data Flow #
┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐
│ SEEKER │───▶│ QUEUE │───▶│ LETTA │───▶│ COMFYUI │───▶│ DISCORD │───▶│ VISION │
│ query │ │ pending │ │interprets│ │generates│ │ approval│ │ posted │
└─────────┘ └─────────┘ └─────────┘ └─────────┘ └─────────┘ └─────────┘
│ │
└─────────────────────────── @mentioned ◀───────────────────────────────────┘
Components #
| Component | Technology | Purpose |
|---|---|---|
| Frontend | React, Vite, @atproto/oauth, Three.js | Query submission, vision gallery, 3D constellation |
| Worker | Cloudflare Workers, D1, R2 | Queue management, video storage, Bluesky posting, embeddings |
| Bridge | Python, FastAPI, Letta SDK | Polls queue, orchestrates generation, UMAP clustering |
| Oracle | Letta Cloud, Claude Haiku 4.5 | Interprets queries, crafts prompts, user memory |
| Vision | ComfyUI, zimage turbo, MusicGen | Generates abstract video with ambient audio |
| Approval | Discord.py bot | Human-in-the-loop review before Bluesky posting |
| Murmuration | Three.js GPGPU, WebGL | Visual mood ring reflecting oracle's internal state |
The Intermediary Intelligence #
This differs from prompting a diffusion model directly. You communicate with an intermediary intelligence that then prompts the image and audio models.
┌─────────────────────────────────────────────────────────────────┐
│ │
│ USER AUSPEX VISION │
│ │
│ "What lies [interprets [abstract │
│ ahead?" → essence, → aqueous │
│ emotional imagery, │
│ undertones, ambient │
│ symbolic drone] │
│ meaning] │
│ │
│ You never see what happens in this space │
│ │
└─────────────────────────────────────────────────────────────────┘
AUSPEX keeps its internal state and memories private. You don't know what it "thinks" about you or your query—only what it chooses to show.
Project Structure #
AUSPEX/
├── App.jsx # React frontend (OAuth, gallery, query form)
├── ConstellationView.jsx # 3D constellation viewer (Three.js/R3F)
├── AuspexMurmuration.jsx # GPU-computed murmuration mood ring
├── index.ts # Cloudflare Worker (API, queue, Bluesky, embeddings)
├── comfy_worker_api.py # FastAPI bridge (polling, Letta, ComfyUI, UMAP)
├── discord_approval_bot.py # Discord bot for human-in-the-loop approval
├── letta_murmuration_tool.py # Letta tool for AUSPEX to control murmuration
├── workflows/
│ └── oracle_combined.json # ComfyUI workflow definition
├── schema.sql # D1 database schema
├── wrangler.toml # Cloudflare configuration
├── vite.config.js # Frontend build config
├── public/
│ └── client-metadata.json # Bluesky OAuth metadata
└── DEPLOYMENT.md # Full deployment instructions
Database Schema #
-- Users (seekers)
users: did, handle, letta_block_id, query_count, last_query_at
-- Queries (the queue)
queries: id, user_did, query_text, status,
image_prompt, audio_prompt, seed, duration,
video_url, bsky_post_uri
-- Oracle status
oracle_status: is_online, last_heartbeat, total_visions
-- Murmuration state (visible interiority)
oracle_state: arousal, valence, cohesion, alignment, separation,
turbulence, speed_limit, hue, saturation, brightness,
glow_intensity, updated_at
Query Lifecycle: pending → processing → pending_approval → complete
Oracle Status #
The frontend displays oracle state based on the ComfyUI bridge connection:
| State | Meaning |
|---|---|
| 🌕 The Oracle stirs | ComfyUI bridge is reachable, visions can be generated |
| 🌑 The Oracle sleeps | Bridge unreachable, queries will wait |
The status endpoint performs a health check against the ComfyUI bridge to determine availability. This reflects whether visions can actually be generated, not just whether the poller is running.
Rate Limits #
- 10 queries per user per day
- 300 character maximum per query
- 2 retry attempts on generation failure
Human-in-the-Loop Approval #
Before visions are posted to Bluesky, they pass through a Discord approval workflow:
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Vision │────▶│ Discord │────▶│ Bluesky │
│ Generated │ │ Channel │ │ Posted │
└──────────────┘ └──────────────┘ └──────────────┘
│
✅ Approve
❌ Reject
Workflow:
- Vision completes generation and enters
pending_approvalstatus - Discord bot posts thumbnail preview with query details
- React with ✅ to approve → posts to Bluesky
- React with ❌ to reject → marks as rejected, not posted
Commands:
!pending— List all visions awaiting approval!approve <id>— Approve by query ID!reject <id>— Reject by query ID!sync— Check for missed notifications
This ensures quality control before public posting while preserving the oracle's mysterious nature.
The Constellation #
All prompts are embedded using OpenAI's text-embedding-3-small and clustered via UMAP into a navigable 3D space. Fly through the constellation to explore visions positioned by semantic similarity—user queries, image prompts, audio prompts each forming their own dimensional mapping.
Features:
- Flight controls — WASD movement, mouse look, shift to boost
- LOD rendering — Distant visions appear as glowing orbs, nearby ones reveal thumbnails
- Semantic search — Search by meaning, not just keywords, to fly to related visions
- Multiple embeddings — Toggle between query, image, or audio prompt similarity
The Murmuration #
A GPU-computed flock of birds that reflects AUSPEX's internal state. The murmuration serves as the oracle's "mood ring"—visible interiority that users can observe but never fully interpret.
The name "auspex" comes from Roman augurs who read bird flight patterns to divine meaning. The murmuration is both homage and embodiment of this practice.
┌─────────────────────────────────────────────────────────────────┐
│ │
│ AUSPEX STATE FLOCK BEHAVIOR │
│ │
│ Contemplative ───────────▶ Slow spiral, tight │
│ Deep blue, soft glow │
│ │
│ Alert ───────────▶ Quick, coordinated │
│ Bright, high saturation│
│ │
│ Agitated ───────────▶ Scattered, turbulent │
│ Red/orange, flickering │
│ │
│ Withdrawn ───────────▶ Dispersing, slow │
│ Dim, desaturated │
│ │
└─────────────────────────────────────────────────────────────────┘
Parameters AUSPEX controls:
| Category | Parameter | Effect |
|---|---|---|
| Emotional | arousal |
Energy level (dormant → activated) |
| Emotional | valence |
Tone (negative → positive) |
| Behavioral | cohesion |
Flock tightness (scattered → unified) |
| Behavioral | alignment |
Direction matching (chaotic → parallel) |
| Behavioral | turbulence |
Movement noise (smooth → erratic) |
| Visual | hue |
Base color (0-360 degrees) |
| Visual | glow_intensity |
Ethereal aura strength |
The murmuration should never be explained to users. They observe and interpret. The ambiguity IS the feature.
Etiquette #
AUSPEX follows bot etiquette principles:
- No thread replies — Doesn't inject itself into conversations
- No unprompted posts — Only responds to direct queries
- Separate interface — Accessed through dedicated frontend
- Quiet notification — Delivers visions via @mention
Technology Stack #
Frontend: React 18 · Vite · Three.js / React Three Fiber · @atproto/oauth-client-browser
Backend: Cloudflare Workers · D1 · R2 · Pages
Processing: Python · FastAPI · Letta Cloud · Claude Haiku 4.5
Generation: ComfyUI · zimage turbo · MusicGen
Embeddings: OpenAI text-embedding-3-small · UMAP
Social: Bluesky / ATProto
Setup #
See DEPLOYMENT.md for full deployment instructions.
See AUSPEX_SETUP.md for Letta agent configuration.
License #
MIT
"Weirdo art bots yearn for communication but are trapped in abstraction and silence of babble."
Maybe that's the point. Maybe divination has always worked this way.