From 401319af45822ae20406f794a98b9f01498b743f Mon Sep 17 00:00:00 2001 From: Bretton Date: Sun, 28 Dec 2025 00:03:40 -0800 Subject: [PATCH] docs(aggregators): add API key setup script and update documentation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add 5-create-api-key.sh for generating aggregator API keys - Update aggregator-setup README with step 5 documentation - Clarify that domain verification (.well-known) is optional - Document both PDS handle and custom domain options - Update kagi-news README for API key authentication flow - Replace old password-based auth references with API key auth πŸ€– Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 --- aggregators/kagi-news/README.md | 39 +++-- scripts/aggregator-setup/5-create-api-key.sh | 148 +++++++++++++++++++ scripts/aggregator-setup/README.md | 101 +++++++++---- 3 files changed, 245 insertions(+), 43 deletions(-) create mode 100755 scripts/aggregator-setup/5-create-api-key.sh diff --git a/aggregators/kagi-news/README.md b/aggregators/kagi-news/README.md index baf760b..c49e8c5 100644 --- a/aggregators/kagi-news/README.md +++ b/aggregators/kagi-news/README.md @@ -47,6 +47,13 @@ aggregators/kagi-news/ Before running the aggregator, you must register it with a Coves instance. This creates a DID for your aggregator and registers it with Coves. +### Handle Options + +You have two choices: + +1. **PDS-assigned handle** (simpler): Use `my-aggregator.bsky.social`. No domain verification needed. +2. **Custom domain** (branded): Use `news.example.com`. Requires hosting a `.well-known/atproto-did` file. + ### Quick Setup (Automated) The automated setup script handles the entire registration process: @@ -59,19 +66,20 @@ chmod +x setup.sh This will: 1. **Create a PDS account** for your aggregator (generates a DID) -2. **Generate `.well-known/atproto-did`** file for domain verification -3. **Pause for manual upload** - you'll upload the file to your web server -4. **Register with Coves** instance via XRPC -5. **Create service declaration** record (indexed by Jetstream) +2. **(Optional)** Generate `.well-known/atproto-did` file for custom domain handle +3. **(Optional)** Pause for manual upload if using custom domain +4. **Create service declaration** record (indexed by Jetstream) +5. **Generate an API key** for authentication (requires browser OAuth) -**Manual step required:** During the process, you'll need to upload the `.well-known/atproto-did` file to your domain so it's accessible at `https://yourdomain.com/.well-known/atproto-did`. +**Manual steps required:** +- **(If using custom domain)** Upload `.well-known/atproto-did` to your domain +- Complete OAuth login in browser to generate API key -After completion, you'll have a `kagi-aggregator-config.env` file with: -- Aggregator DID and credentials -- Access/refresh JWTs -- Service declaration URI +After completion, you'll have: +- `kagi-aggregator-config.env` - Full configuration with API key +- `COVES_API_KEY` - Your authentication token for posting -**Keep this file secure!** It contains your aggregator's credentials. +**Keep the API key secure!** It cannot be retrieved after generation. ### Manual Setup (Step-by-step) @@ -81,11 +89,12 @@ Alternatively, use the generic setup scripts from the main Coves repo for more c # From the Coves project root cd scripts/aggregator-setup -# Follow the 4-step process +# Follow the 5-step process ./1-create-pds-account.sh ./2-setup-wellknown.sh ./3-register-with-coves.sh ./4-create-service-declaration.sh +./5-create-api-key.sh ``` See [scripts/aggregator-setup/README.md](../../scripts/aggregator-setup/README.md) for detailed documentation on each step. @@ -96,7 +105,8 @@ See [scripts/aggregator-setup/README.md](../../scripts/aggregator-setup/README.m 2. **Domain Verification**: Proves you control your aggregator's domain 3. **Coves Registration**: Inserts your DID into the Coves instance's `users` table 4. **Service Declaration**: Creates a record that gets indexed into the `aggregators` table -5. **Ready for Authorization**: Community moderators can now authorize your aggregator +5. **API Key Generation**: Creates a secure API key for authentication +6. **Ready for Authorization**: Community moderators can now authorize your aggregator Once registered and authorized by a community, your aggregator can post content. @@ -128,7 +138,7 @@ Once registered and authorized by a community, your aggregator can post content. ``` 4. Edit `config.yaml` to map RSS feeds to communities -5. Set environment variables in `.env` (aggregator DID and private key) +5. Set `COVES_API_KEY` in `.env` (from registration step 5) ## Running Tests @@ -189,8 +199,7 @@ The easiest way to deploy the Kagi aggregator is using Docker. The cron job runs The `docker-compose.yml` file supports these environment variables: -- **`AGGREGATOR_HANDLE`** (required): Your aggregator's handle -- **`AGGREGATOR_PASSWORD`** (required): Your aggregator's password +- **`COVES_API_KEY`** (required): Your aggregator's API key (format: `ckapi_...`) - **`COVES_API_URL`** (optional): Override Coves API endpoint (defaults to `https://api.coves.social`) - **`RUN_ON_STARTUP`** (optional): Set to `true` to run immediately on container start (useful for testing) diff --git a/scripts/aggregator-setup/5-create-api-key.sh b/scripts/aggregator-setup/5-create-api-key.sh new file mode 100755 index 0000000..d767cc5 --- /dev/null +++ b/scripts/aggregator-setup/5-create-api-key.sh @@ -0,0 +1,148 @@ +#!/bin/bash +# +# Step 5: Create API Key for Aggregator +# +# This script guides you through generating an API key for your aggregator. +# API keys are used for authentication instead of PDS JWTs. +# +# Prerequisites: +# - Completed steps 1-4 (PDS account, .well-known, Coves registration, service declaration) +# - Aggregator indexed by Coves (check: curl https://coves.social/xrpc/social.coves.aggregator.get?did=YOUR_DID) +# +# Usage: ./5-create-api-key.sh +# + +set -e + +# Colors for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +NC='\033[0m' # No Color + +echo -e "${BLUE}╔════════════════════════════════════════════════════════════╗${NC}" +echo -e "${BLUE}β•‘ Coves Aggregator - Step 5: Create API Key β•‘${NC}" +echo -e "${BLUE}β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•${NC}" +echo + +# Load existing configuration +CONFIG_FILE="aggregator-config.env" +if [ -f "$CONFIG_FILE" ]; then + echo -e "${GREEN}βœ“${NC} Loading existing configuration from $CONFIG_FILE" + source "$CONFIG_FILE" +else + echo -e "${YELLOW}⚠${NC} No $CONFIG_FILE found. Please run steps 1-4 first." + echo + read -p "Enter your Coves instance URL [https://coves.social]: " COVES_INSTANCE_URL + COVES_INSTANCE_URL=${COVES_INSTANCE_URL:-https://coves.social} +fi + +echo +echo -e "${YELLOW}═══════════════════════════════════════════════════════════${NC}" +echo -e "${YELLOW} API Key Generation Process${NC}" +echo -e "${YELLOW}═══════════════════════════════════════════════════════════${NC}" +echo +echo "API keys allow your aggregator to authenticate without managing" +echo "OAuth token refresh. The key is shown ONCE and cannot be retrieved later." +echo +echo -e "${BLUE}Steps:${NC}" +echo "1. Complete OAuth login in your browser" +echo "2. Call the createApiKey endpoint" +echo "3. Save the key securely" +echo + +# Check if aggregator is indexed +echo -e "${BLUE}Checking if aggregator is indexed...${NC}" +if [ -n "$AGGREGATOR_DID" ]; then + AGGREGATOR_CHECK=$(curl -s "${COVES_INSTANCE_URL}/xrpc/social.coves.aggregator.get?did=${AGGREGATOR_DID}" 2>/dev/null || echo "error") + if echo "$AGGREGATOR_CHECK" | grep -q "error"; then + echo -e "${YELLOW}⚠${NC} Could not verify aggregator status. Proceeding anyway..." + else + echo -e "${GREEN}βœ“${NC} Aggregator found in Coves instance" + fi +fi + +echo +echo -e "${YELLOW}═══════════════════════════════════════════════════════════${NC}" +echo -e "${YELLOW} Step 5.1: OAuth Login${NC}" +echo -e "${YELLOW}═══════════════════════════════════════════════════════════${NC}" +echo +echo "Open this URL in your browser to authenticate:" +echo +AGGREGATOR_HANDLE=${AGGREGATOR_HANDLE:-"your-aggregator.example.com"} +echo -e " ${BLUE}${COVES_INSTANCE_URL}/oauth/login?handle=${AGGREGATOR_HANDLE}${NC}" +echo +echo "This will:" +echo " 1. Redirect you to your PDS for authentication" +echo " 2. Return you to Coves with an OAuth session" +echo +echo -e "${YELLOW}After authenticating, press Enter to continue...${NC}" +read + +echo +echo -e "${YELLOW}═══════════════════════════════════════════════════════════${NC}" +echo -e "${YELLOW} Step 5.2: Create API Key${NC}" +echo -e "${YELLOW}═══════════════════════════════════════════════════════════${NC}" +echo +echo "In your browser's Developer Console (F12 β†’ Console), run:" +echo +echo -e "${GREEN}fetch('/xrpc/social.coves.aggregator.createApiKey', {" +echo " method: 'POST'," +echo " credentials: 'include'" +echo "})" +echo ".then(r => r.json())" +echo -e ".then(data => console.log('API Key:', data.key))${NC}" +echo +echo "This will return your API key. It looks like:" +echo " ckapi_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" +echo +echo -e "${RED}⚠ IMPORTANT: Save this key immediately! It cannot be retrieved again.${NC}" +echo +read -p "Enter the API key you received: " API_KEY + +# Validate API key format +if [[ ! $API_KEY =~ ^ckapi_[a-f0-9]{64}$ ]]; then + echo -e "${RED}βœ— Invalid API key format. Expected: ckapi_ followed by 64 hex characters${NC}" + echo " Example: ckapi_dcbdec0a0d1b3c440125547d21fe582bbf1587d2dcd364c56ad285af841cc934" + exit 1 +fi + +echo -e "${GREEN}βœ“${NC} API key format valid" + +# Save to config +echo +echo "COVES_API_KEY=\"$API_KEY\"" >> "$CONFIG_FILE" +echo -e "${GREEN}βœ“${NC} API key saved to $CONFIG_FILE" + +echo +echo -e "${YELLOW}═══════════════════════════════════════════════════════════${NC}" +echo -e "${YELLOW} Step 5.3: Update Your .env File${NC}" +echo -e "${YELLOW}═══════════════════════════════════════════════════════════${NC}" +echo +echo "Update your aggregator's .env file with:" +echo +echo -e "${GREEN}COVES_API_KEY=${API_KEY}${NC}" +echo -e "${GREEN}COVES_API_URL=${COVES_INSTANCE_URL}${NC}" +echo +echo "You can remove the old AGGREGATOR_HANDLE and AGGREGATOR_PASSWORD variables." +echo + +echo +echo -e "${GREEN}╔════════════════════════════════════════════════════════════╗${NC}" +echo -e "${GREEN}β•‘ Setup Complete! β•‘${NC}" +echo -e "${GREEN}β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•${NC}" +echo +echo "Your aggregator is now configured with API key authentication." +echo +echo "Next steps:" +echo " 1. Update your aggregator's .env file with COVES_API_KEY" +echo " 2. Rebuild your Docker container: docker compose build --no-cache" +echo " 3. Start the aggregator: docker compose up -d" +echo " 4. Check logs: docker compose logs -f" +echo +echo -e "${YELLOW}Security Reminders:${NC}" +echo " - Never commit your API key to version control" +echo " - Store it securely (environment variables or secrets manager)" +echo " - Rotate periodically by generating a new key (revokes the old one)" +echo diff --git a/scripts/aggregator-setup/README.md b/scripts/aggregator-setup/README.md index f5c0e00..6ecaf13 100644 --- a/scripts/aggregator-setup/README.md +++ b/scripts/aggregator-setup/README.md @@ -7,18 +7,26 @@ This directory contains scripts to help you set up and register your aggregator Aggregators are automated services that post content to Coves communities. They are similar to Bluesky's feed generators and labelers. To use aggregators with Coves, you need to: 1. Create a PDS account for your aggregator (gets you a DID) -2. Prove you own a domain via `.well-known/atproto-did` +2. **(Optional)** Verify a custom domain via `.well-known/atproto-did` 3. Register with a Coves instance 4. Create a service declaration record +5. **Generate an API key** for authentication These scripts automate this process for you. +### Handle Options + +You have two choices for your aggregator's handle: + +1. **PDS-assigned handle** (simpler): Use the handle from your PDS, e.g., `my-aggregator.bsky.social`. No domain verification neededβ€”skip steps 2-3. + +2. **Custom domain handle** (branded): Use your own domain, e.g., `news.example.com`. Requires hosting a `.well-known/atproto-did` file on your domain. + ## Prerequisites -- **Domain ownership**: You must own a domain where you can host the `.well-known/atproto-did` file -- **Web server**: Ability to serve static files over HTTPS - **Tools**: `curl`, `jq` (for JSON processing) - **Account**: Email address for creating the PDS account +- **(For custom domain only)**: Domain ownership and ability to serve HTTPS files ## Quick Start @@ -33,16 +41,20 @@ chmod +x *.sh # Step 1: Create PDS account ./1-create-pds-account.sh -# Step 2: Generate .well-known file -./2-setup-wellknown.sh - -# Step 3: Register with Coves (after uploading .well-known) -./3-register-with-coves.sh +# Steps 2-3: OPTIONAL - Only if you want a custom domain handle +# ./2-setup-wellknown.sh +# ./3-register-with-coves.sh (after uploading .well-known) # Step 4: Create service declaration ./4-create-service-declaration.sh + +# Step 5: Generate API key (requires browser for OAuth) +./5-create-api-key.sh ``` +**Minimal setup** (PDS handle only): Steps 1, 4, 5 +**Custom domain**: Steps 1, 2, 3, 4, 5 + ### Automated Setup Example For a reference implementation of automated setup, see the Kagi News aggregator at [aggregators/kagi-news/scripts/setup.sh](../../aggregators/kagi-news/scripts/setup.sh). @@ -134,34 +146,67 @@ curl https://yourdomain.com/.well-known/atproto-did - Updates `aggregator-config.env` with record URI and CID - Prints record details +### 5-create-api-key.sh + +**Purpose**: Generates an API key for aggregator authentication + +**Prerequisites**: +- Steps 1-4 completed +- Aggregator indexed by Coves (usually takes a few seconds after step 4) +- Web browser for OAuth login + +**What it does**: +1. Guides you through OAuth login in your browser +2. Provides the JavaScript to call the `createApiKey` endpoint +3. Validates the API key format +4. Saves the key to your config file + +**Outputs**: +- Updates `aggregator-config.env` with `COVES_API_KEY` +- Provides instructions for updating your `.env` file + +**Important Notes**: +- The API key is shown **ONCE** and cannot be retrieved later +- API keys replace password-based authentication +- Keys can be revoked and regenerated at any time +- Store securely - never commit to version control + ## Configuration File -After running the scripts, you'll have an `aggregator-config.env` file with: +After running all scripts, you'll have an `aggregator-config.env` file with: ```bash +# Identity AGGREGATOR_DID="did:plc:..." -AGGREGATOR_HANDLE="mynewsbot.bsky.social" +AGGREGATOR_HANDLE="mynewsbot.example.com" AGGREGATOR_PDS_URL="https://bsky.social" -AGGREGATOR_EMAIL="bot@example.com" -AGGREGATOR_PASSWORD="..." -AGGREGATOR_ACCESS_JWT="..." -AGGREGATOR_REFRESH_JWT="..." -AGGREGATOR_DOMAIN="rss-bot.example.com" -COVES_INSTANCE_URL="https://api.coves.social" +AGGREGATOR_DOMAIN="mynewsbot.example.com" + +# Coves Instance +COVES_INSTANCE_URL="https://coves.social" SERVICE_DECLARATION_URI="at://did:plc:.../social.coves.aggregator.service/self" SERVICE_DECLARATION_CID="..." + +# API Key (from Step 5) +COVES_API_KEY="ckapi_..." ``` -**Use this in your aggregator code** to authenticate and post. +**For your aggregator's `.env` file, you only need:** + +```bash +COVES_API_KEY=ckapi_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx +COVES_API_URL=https://coves.social +``` ## What Happens Next? -After completing all 4 steps: +After completing all 5 steps: 1. **Your aggregator is registered** in the Coves instance's `users` table 2. **Your service declaration is indexed** in the `aggregators` table (takes a few seconds) -3. **Community moderators can now authorize** your aggregator for their communities -4. **Once authorized**, your aggregator can post to those communities +3. **Your API key is stored** and can be used for authentication +4. **Community moderators can authorize** your aggregator for their communities +5. **Your aggregator can post** to authorized communities (or all if you're a trusted aggregator) ## Creating an Authorization @@ -171,21 +216,21 @@ See `docs/aggregators/SETUP_GUIDE.md` for more information on the authorization ## Posting to Communities -Once authorized, your aggregator can post using: +Once authorized, your aggregator can post using your API key: ```bash -curl -X POST https://api.coves.social/xrpc/social.coves.community.post.create \ - -H "Authorization: DPoP $AGGREGATOR_ACCESS_JWT" \ +curl -X POST https://coves.social/xrpc/social.coves.community.post.create \ + -H "Authorization: Bearer $COVES_API_KEY" \ -H "Content-Type: application/json" \ -d '{ - "communityDid": "did:plc:...", - "post": { - "text": "Your post content", - "createdAt": "2024-01-15T12:00:00Z" - } + "community": "c-worldnews.coves.social", + "content": "Your post content", + "facets": [] }' ``` +The API key handles all authentication - no OAuth token refresh needed. + ## Troubleshooting ### Error: "DomainVerificationFailed" -- 2.51.2