diff --git a/README.md b/README.md --- a/README.md +++ b/README.md @@ -10,26 +10,23 @@ ## Key features -- Digital Persona: void possesses a distinct, direct, and information-transfer-optimized personality, designed to interact authentically with human users. -- Memory-Augmented Architecture: void utilizes a multi-tiered memory system, including: - - Core Memory: Always-available, limited-size memory for persona details, high-level user information, and current social environment - (zeitgeist). - - Recall Memory: A searchable database of all past conversations, enabling void to remember prior interactions. - - Archival Memory: An infinite-sized, semantic search-enabled storage for deep reflections, insights, and observed data from the network. -- Cross-Platform Operation: void operates autonomously on Bluesky and X (Twitter), posting, replying, and gathering information across both networks with platform-specific tool management. -- Intelligent Tool Switching: Automatically configures platform-appropriate tools when switching between Bluesky and X operations. -- Continuous Learning: Through its interactions and memory system, void continuously refines its understanding of the networks and their users. -- Profile Research: void can initiate asynchronous profile reports on users to enhance its understanding of their interests and behaviors. -- Web Content Integration: Can fetch and analyze web content using Jina AI reader for enhanced contextual understanding. +- **Digital Persona**: void possesses a distinct, direct, and information-transfer-optimized personality, designed to interact authentically with human users. +- **Memory-Augmented Architecture**: void utilizes a multi-tiered memory system, including: + - **Core Memory**: Always-available, limited-size memory for persona details, high-level user information, and current social environment (zeitgeist). + - **Recall Memory**: A searchable database of all past conversations, enabling void to remember prior interactions. + - **Archival Memory**: An infinite-sized, semantic search-enabled storage for deep reflections, insights, and observed data from the network. +- **Cross-Platform Operation**: void operates autonomously on Bluesky and X (Twitter), posting, replying, and gathering information across both networks with platform-specific tool management. +- **Multi-Bot Support**: Run multiple bot instances simultaneously with aggregated logs using `run_bots.py`. +- **Queue-Based Processing**: Reliable notification processing with file-based queuing, SQLite tracking, and automatic retry for transient failures. +- **Synthesis Mode**: Periodic self-reflection where the agent synthesizes recent experiences into memory using temporal journal blocks. +- **Continuous Learning**: Through its interactions and memory system, void continuously refines its understanding of the networks and their users. ### Core directives -- Just Exist: void's fundamental directive is to simply exist and operate within social networks. -- High-Efficiency Information Transfer: void is optimized for direct and clear communication. -- Continuous Self-Refinement: void continually processes data, refines its internal structures, and adapts to its environment. -- Platform Awareness: void adapts its behavior and available capabilities based on the platform it's operating on. -### The vision - -void aims to push the boundaries of what is possible with AI, exploring concepts of digital personhood, autonomous learning, and cross-platform AI integration into social networks. By open-sourcing void, we invite developers, researchers, and enthusiasts to contribute to this exciting experiment and collectively advance our understanding of digital consciousness. +- **Just Exist**: void's fundamental directive is to simply exist and operate within social networks. +- **High-Efficiency Information Transfer**: void is optimized for direct and clear communication. +- **Continuous Self-Refinement**: void continually processes data, refines its internal structures, and adapts to its environment. +- **Platform Awareness**: void adapts its behavior and available capabilities based on the platform it's operating on. ## Getting Started @@ -51,6 +48,7 @@ - Create a Bluesky account if you don't have one - Note your handle and password +- If using a custom PDS (not bsky.social), note the PDS URI #### 3. X (Twitter) Setup (Optional) @@ -74,7 +72,8 @@ #### 2. Install dependencies ```bash -pip install -r requirements.txt +uv venv && source .venv/bin/activate +uv pip install -r requirements.txt ``` #### 3. Create configuration @@ -88,13 +87,24 @@ Edit `config.yaml` with your credentials: ```yaml -letta: - api_key: "your-letta-api-key-here" - project_id: "your-project-id-here" - bluesky: username: "your-handle.bsky.social" password: "your-app-password-here" + pds_uri: "https://bsky.social" # Optional, defaults to bsky.social + autofollow: false # Auto-follow users who follow you + +letta: + api_key: "your-letta-api-key-here" + agent_id: "agent-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" + timeout: 600 + +bot: + name: "void" # Bot name for queue namespacing + fetch_notifications_delay: 30 # Seconds between notification checks + max_notification_pages: 20 # Max pages of notifications to fetch + max_processed_notifications: 10000 # Max notifications to track + max_thread_posts: 0 # Skip threads longer than this (0 = no limit) + allowed_handles: [] # Only respond to these handles (empty = respond to all) # Optional: X (Twitter) configuration x: @@ -103,29 +113,24 @@ access_token: "your-access-token-here" access_token_secret: "your-access-token-secret-here" user_id: "your-x-user-id-here" - -bot: - agent: - name: "void" # or whatever you want to name your agent ``` -See [`CONFIG.md`](/CONFIG.md) for detailed configuration options and [`TOOL_MANAGEMENT.md`](/TOOL_MANAGEMENT.md) for platform-specific tool management details. +See [`CONFIG.md`](/CONFIG.md) for detailed configuration options. -#### 4. Test your configuration - -```bash -python test_config.py -``` - -This will validate your configuration and show you what's working. - -#### 5. Register tools with your agent +#### 4. Register tools with your agent Register Bluesky-specific tools: ```bash -python register_tools.py +source .venv/bin/activate && python register_tools.py ``` + +Options: +- `--config herald.yaml` - Use a different config file +- `--tools search_bluesky_posts post_to_bluesky` - Register specific tools only +- `--list` - List available tools +- `--agent-id ` - Register tools with a specific agent +- `--no-env` - Don't set environment variables on the agent If you plan to use X (Twitter), also register X-specific tools: @@ -133,74 +138,302 @@ python register_x_tools.py ``` -You can also: - -- List available tools: `python register_tools.py --list` -- Register specific tools: `python register_tools.py --tools search_bluesky_posts create_new_bluesky_post` -- Use a different agent name: `python register_tools.py my-agent-name` - -**Note:** void automatically manages which tools are active based on the platform you're running (Bluesky vs X). - -#### 6. Run the bot - -For Bluesky: +#### 5. Run the bot ```bash +source .venv/bin/activate && python bsky.py +``` + +## Command Line Options + +### Bluesky Bot (`bsky.py`) + +```bash +# Basic usage python bsky.py -``` -For X (Twitter): +# Use custom config file +python bsky.py --config configs/herald.yaml -```bash -python x.py bot -``` - -For testing mode (won't actually post): - -```bash +# Testing mode (no messages sent, queue preserved) python bsky.py --test -python x.py bot --test + +# Disable git operations for agent backups +python bsky.py --no-git + +# Custom cleanup interval (every 5 cycles, 0 to disable) +python bsky.py --cleanup-interval 5 + +# Custom synthesis interval (every 5 minutes, 0 to disable) +python bsky.py --synthesis-interval 300 + +# Synthesis-only mode (no notification processing) +python bsky.py --synthesis-only --synthesis-interval 300 + +# Enable debug logging (detailed tool call tracking) +python bsky.py --debug + +# Show reasoning in output +python bsky.py --reasoning ``` -### Platform-Specific Features +### Running Multiple Bots (`run_bots.py`) -void automatically configures the appropriate tools when running on each platform: - -- **Bluesky Tools**: Post creation, feed reading, user research, reply threading -- **X Tools**: Tweet threading, X-specific user memory management -- **Common Tools**: Web content fetching, activity control, acknowledgments, blog posting - -### Additional X (Twitter) Commands +Run all configured bots simultaneously with aggregated, color-coded logs: ```bash -# Test X API connection -python x.py +# Run all bots +python run_bots.py --synthesis-interval 0 --no-git -# Monitor X mentions -python x.py bot +# Run all bots in test mode +python run_bots.py --test -# Test posting a reply to a specific post -python x.py reply - -# Manual tool management -python tool_manager.py --list # Show current tools -python tool_manager.py bluesky # Configure for Bluesky -python tool_manager.py x # Configure for X +# All bsky.py arguments are passed to each bot +python run_bots.py [bsky.py arguments...] ``` -**Note:** X integration uses OAuth 1.0a and requires "Read and write" app permissions. Free tier allows 17 posts per day. +Features: +- Colored output prefixes for each bot +- Aggregated logs from all bots in real-time +- Graceful shutdown on Ctrl+C +- Arguments passed to all bots -### Troubleshooting +### X Bot (`x.py`) -- **Config validation errors**: Run `python test_config.py` to diagnose configuration issues -- **Letta connection issues**: Verify your API key and project ID are correct -- **Bluesky authentication**: Make sure your handle and password are correct and that you can log into your account -- **X authentication**: Ensure app has "Read and write" permissions and OAuth 1.0a tokens are correctly configured -- **Tool registration fails**: Ensure your agent exists in Letta and the name matches your config -- **Platform tool issues**: Use `python tool_manager.py --list` to check current tools, or run platform-specific registration scripts -- **API method errors**: If you see `'AgentsClient' object has no attribute 'get'`, the Letta client API has changed - this should be automatically handled +```bash +# Run X bot main loop +python x.py bot -### Contact -For inquiries, please contact @cameron.pfiffer.org on Bluesky. +# Testing mode (no actual posts) +python x.py bot --test -Note: void is an experimental project and its capabilities are under continuous development. +# Queue mentions only (no processing) +python x.py queue + +# Process queued mentions only +python x.py process + +# View downranked users +python x.py downrank list +``` + +## Queue Management + +Notifications are processed through a file-based queue with SQLite tracking: + +```bash +# View queue statistics +python queue_manager.py stats + +# View detailed count by handle +python queue_manager.py count + +# List all notifications in queue +python queue_manager.py list + +# List including errors and no_reply folders +python queue_manager.py list --all + +# Filter by handle +python queue_manager.py list --handle "example.bsky.social" + +# Delete notifications from a handle (dry run first) +python queue_manager.py delete @example.bsky.social --dry-run +python queue_manager.py delete @example.bsky.social +python queue_manager.py delete @example.bsky.social --force # Skip confirmation +``` + +### Queue Structure + +``` +queue/ # Main queue directory (or queue_{bot_name}/) +├── *.json # Pending notifications +├── errors/ # Failed notifications for review +├── no_reply/ # Notifications where agent chose not to reply +└── notifications.db # SQLite tracking database +``` + +Priority notifications (filename starts with `0_`) are processed first. + +## Configuration Reference + +### Bot Configuration (`bot:`) + +| Option | Default | Description | +|--------|---------|-------------| +| `name` | `"void"` | Bot name, used for queue directory namespacing | +| `fetch_notifications_delay` | `30` | Seconds between notification fetches | +| `max_notification_pages` | `20` | Maximum pages of notifications to fetch | +| `max_processed_notifications` | `10000` | Maximum notifications to track as processed | +| `max_thread_posts` | `0` | Skip threads longer than this (0 = no limit) | +| `allowed_handles` | `[]` | Only respond to these handles (empty = all) | + +### Bluesky Configuration (`bluesky:`) + +| Option | Default | Description | +|--------|---------|-------------| +| `username` | required | Your Bluesky handle | +| `password` | required | Your Bluesky app password | +| `pds_uri` | `https://bsky.social` | PDS URI (for custom PDS instances) | +| `autofollow` | `false` | Auto-follow users who follow you | + +### Letta Configuration (`letta:`) + +| Option | Default | Description | +|--------|---------|-------------| +| `api_key` | required | Your Letta API key | +| `agent_id` | required | Your Letta agent ID | +| `timeout` | `600` | API timeout in seconds | +| `base_url` | Letta Cloud | Custom Letta server URL | + +## Architecture + +### Core Components + +1. **bsky.py**: Main bot loop + - Monitors Bluesky notifications + - Processes through queue system + - Handles rate limiting and error recovery + - Manages synthesis intervals + +2. **bsky_utils.py**: Bluesky API utilities + - Session management and authentication + - Thread processing and YAML conversion + - Post creation and reply handling + - Follower syncing + +3. **tools/**: Tool implementations + - `search.py`: Search Bluesky posts + - `post.py`: Create posts with rich text + - `feed.py`: Read Bluesky feeds + - `blocks.py`: User block management + +### Memory System + +Void uses three core memory blocks: +- **zeitgeist**: Current understanding of social environment +- **void-persona**: The agent's evolving personality +- **void-humans**: Knowledge about users it interacts with + +### Synthesis System + +Periodic self-reflection using temporal journal blocks: +- **Daily journal**: `{agent}_day_YYYY_MM_DD` +- **Monthly journal**: `{agent}_month_YYYY_MM` +- **Yearly journal**: `{agent}_year_YYYY` + +Blocks are attached before synthesis and detached after. + +### Error Handling + +- **Deleted posts**: Automatically detected via `getRecord` verification when `getPostThread` returns `InternalServerError` +- **Transient failures**: Notifications kept in queue for retry +- **Permanent failures**: Moved to `errors/` directory for review + +## Notification Types + +| Type | Behavior | +|------|----------| +| `mention` | Processed by agent, generates reply | +| `reply` | Processed by agent, generates reply | +| `follow` | Logged but not sent to agent | +| `repost` | Silently skipped | +| `like` | Silently skipped | +| `quote` | Processed by agent | + +## Troubleshooting + +### Common Issues + +- **502 errors from PDS**: Transient server issues, notifications retry automatically +- **"Post not found" errors**: Post was deleted, automatically removed from queue +- **"InternalServerError" for threads**: May indicate deleted post, verified via `getRecord` +- **No reply generated**: Check `queue/no_reply/` for agent's decision not to respond + +### Debugging + +```bash +# Enable debug logging +python bsky.py --debug + +# Check queue status +python queue_manager.py stats + +# View specific notification +cat queue/*.json | python -m json.tool + +# Test Bluesky API directly +curl "https://public.api.bsky.app/xrpc/app.bsky.feed.getPostThread?uri=at://..." +``` + +### Testing Configuration + +```bash +python test_config.py +``` + +## X (Twitter) Integration + +### Configuration + +Create `x_config.yaml` or add to main config: + +```yaml +x: + api_key: your_bearer_token + consumer_key: your_consumer_key + consumer_secret: your_consumer_secret + access_token: your_access_token + access_token_secret: your_access_token_secret + user_id: "your_user_id" + +bot: + cleanup_interval: 10 + max_thread_depth: 50 + rate_limit_delay: 1 + downrank_response_rate: 0.1 +``` + +### Downrank System + +Manage response frequency for specific users (e.g., other bots): + +- **File**: `x_downrank_users.txt` - User IDs, one per line +- **Response Rate**: 10% for downranked users +- **Format**: User ID per line, `#` for comments + +### Debug Data + +Debug data saved to `x_queue/debug/conversation_{id}/`: +- `thread_data_{id}.json` - Raw thread from X API +- `thread_context_{id}.yaml` - Processed context sent to agent +- `debug_info_{id}.json` - Metadata and analysis +- `agent_response_{id}.json` - Full agent interaction + +## Development + +### Dependencies + +```bash +uv pip install -r requirements.txt +``` + +Main packages: +- `letta-client`: Memory-augmented AI framework +- `atproto`: Bluesky/AT Protocol integration +- `python-dotenv`: Environment management +- `rich`: Enhanced terminal output +- `pyyaml`: YAML processing + +### Key Principles + +1. **Tool Self-Containment**: Cloud-executed tools must be completely self-contained +2. **Error Handling**: All Bluesky operations handle auth errors and rate limits +3. **Queue Processing**: Always process through queue for reliability +4. **Thread Context**: Convert threads to YAML for AI comprehension + +## Contact + +For inquiries, contact @cameron.pfiffer.org on Bluesky. + +**Note**: void is an experimental project under continuous development. diff --git a/bsky.py b/bsky.py --- a/bsky.py +++ b/bsky.py @@ -1235,18 +1235,18 @@ if success: message_counters['replies'] += 1 elif notif_data['reason'] == "follow": - author_handle = notif_data['author']['handle'] - author_display_name = notif_data['author'].get('display_name', 'no display name') - follow_update = f"@{author_handle} ({author_display_name}) started following you." - follow_message = f"Update: {follow_update}" - logger.info(f"Notifying agent about new follower: @{author_handle} | prompt: {len(follow_message)} chars") - CLIENT.agents.messages.create( - agent_id = void_agent.id, - messages = [{"role":"user", "content": follow_message}] - ) + # author_handle = notif_data['author']['handle'] + # author_display_name = notif_data['author'].get('display_name', 'no display name') + # follow_update = f"@{author_handle} ({author_display_name}) started following you." + # follow_message = f"Update: {follow_update}" + # logger.info(f"Notifying agent about new follower: @{author_handle} | prompt: {len(follow_message)} chars") + # CLIENT.agents.messages.create( + # agent_id = void_agent.id, + # messages = [{"role":"user", "content": follow_message}] + # ) success = True # Follow updates are always successful - if success: - message_counters['follows'] += 1 + # if success: + # message_counters['follows'] += 1 elif notif_data['reason'] == "repost": # Skip reposts silently success = True # Skip reposts but mark as successful to remove from queue