Wisp Hosting Service #

Minimal microservice for hosting static sites from the AT Protocol. Built with Hono and Bun.

Features #

• Custom Domain Hosting: Serve verified custom domains
• Wisp.place Subdomains: Serve registered *.wisp.place subdomains
• DNS Hash Routing: Support DNS verification via hash.dns.wisp.place
• Direct File Serving: Access sites via sites.wisp.place/:identifier/:site/* (no DB lookup)
• Automatic Caching: Reads immutable site objects from the configured cold tier and promotes into bounded hot/warm caches; firehose updates invalidate mutable upper caches
• SSRF Protection: Hardened fetch with timeout, size limits, and private IP blocking

Routes #

1. Custom Domains (/*)

   • Serves verified custom domains (example.com)
   • DB lookup: custom_domains table

2. Wisp Subdomains (/*.wisp.place/*)

   • Serves registered subdomains (alice.wisp.place)
   • DB lookup: domains table

3. DNS Hash Routing (/hash.dns.wisp.place/*)

   • DNS verification routing for custom domains
   • DB lookup: custom_domains by hash

4. Direct Serving (/sites.wisp.place/:identifier/:site/*)

   • Direct access without DB lookup
   • :identifier can be DID or handle
   • Reads from cold storage when not cached; storage misses enqueue firehose revalidation rather than downloading inside the request
   • Automatic HTML path rewriting: Absolute paths (/style.css) are rewritten to relative paths (sites.wisp.place/:identifier/:site/style.css)

Setup #

# Install dependencies
bun install

# Copy environment file
cp .env.example .env

# Run in development
bun run dev

# Run in production
bun run start

Environment Variables #

• DATABASE_URL - PostgreSQL connection string
• PORT - HTTP server port (default: 3001)
• BASE_HOST - Base domain (default: wisp.place)

Architecture #

• Hono: Minimal web framework
• Postgres: Database for domain/site lookups
• AT Protocol: Decentralized storage
• Redis: Cache invalidation events from the firehose service
• Bun: Runtime and file serving

Cache Structure #

cache/sites/
  did:plc:abc123/
    sitename/
      index.html
      style.css
      assets/
        logo.png

Health Check #

curl http://localhost:3001/health

/health is readiness: HTTP 200 only when the configured Redis subscriber has acknowledged SUBSCRIBE, replay is healthy, and storage read health is acceptable. Degraded readiness returns HTTP 503. /live is dependency-free liveness (HTTP 200). The public payload omits stream cursors, credentials, endpoints and bucket names.

Invalidation lifecycle and overload #

One scoped Effect dispatcher owns the subscriber, replay reader, ordered worker, heartbeat/supervisor and shutdown drain. It disconnects blocking Redis readers before joining accepted work. HTTP drains first; actual uncancelable storage and cursor-file writes finish before databases and exporters close. Partial startup failures unwind already acquired resources. Exporter initialization is awaited.

• WISP_CACHE_INVALIDATION_QUEUE_CAPACITY: queued event count (default 1024; valid range 1–65536). The worker processes one event at a time; replay has at most one pending admission. Pub/sub uses nonblocking admission, never suspended-offer fibers.
• WISP_CACHE_INVALIDATION_MAX_MESSAGE_BYTES: raw payload ceiling (default and maximum 1 MiB). Oversized input is rejected before JSON parsing/queue retention; replay also checks field bytes before reconstructing its payload.
• Only ordered replay advances the cursor. Replay awaits one atomic cursor-file write before exposing progress or clearing a terminal updating marker; there is no per-event detached persistence queue. Persistence/storage failures retain the cursor and retry with degraded readiness.
• Durable pub/sub overflow degrades readiness until replay reaches the greatest missed stream ID. Producers must XADD before PUBLISH. A higher live ID cannot skip earlier replay entries. A stream retention gap clears mutable maps/hot caches, not warm/cold namespaces; warm reads still validate source CIDs.
• Legacy events without a durable stream ID have no replay recovery guarantee. Overflow, unreplayable processing defects, and oversized input latch fail-closed: tenant requests return HTTP 503 with Cache-Control: no-store; /health reports degraded readiness and /live remains available. The safe payload adds overloadLatched, a sanitized overloadReason, and an overflow counter.

An overload latch cannot be cleared by reconnecting or restarting the dispatcher. Operators must reconcile the affected caches/content with the authoritative source before restarting the process. A process restart clears in-memory state but is not proof that the missed legacy invalidation was recovered. Do not wipe cold storage or broad warm-cache namespaces as a shortcut.

HTML Path Rewriting #

When serving sites via the /s/:identifier/:site/* route, HTML files are automatically processed to rewrite absolute paths to work correctly in the subdirectory context.

What gets rewritten:

• src attributes (images, scripts, iframes)
• href attributes (links, stylesheets)
• action attributes (forms)
• poster, data attributes (media)
• srcset attributes (responsive images)

What's preserved:

• External URLs (https://example.com/style.css)
• Protocol-relative URLs (//cdn.example.com/script.js)
• Data URIs (data:image/png;base64,...)
• Anchors (/#section)
• Already relative paths (./style.css, ../images/logo.png)

Example:

<!-- Original HTML -->
<link rel="stylesheet" href="/style.css">
<img src="/images/logo.png">

<!-- Served at /s/did:plc:abc123/mysite/ becomes -->
<link rel="stylesheet" href="sites.wisp.place/did:plc:abc123/mysite/style.css">
<img src="sites.wisp.place/did:plc:abc123/mysite/images/logo.png">

This ensures sites work correctly when served from subdirectories without requiring manual path adjustments.

Security #

SSRF Protection #

All external HTTP requests are protected against Server-Side Request Forgery (SSRF) attacks:

• 5-second timeout on all requests
• Size limits: 1MB for JSON, 10MB default, 100MB for file blobs
• Blocked private IP ranges:
  • Loopback (127.0.0.0/8, ::1)
  • Private networks (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16)
  • Link-local (169.254.0.0/16, fe80::/10)
  • Cloud metadata endpoints (169.254.169.254)
• Protocol validation: Only HTTP/HTTPS allowed
• Streaming with size enforcement: Prevents memory exhaustion from large responses