Monorepo for wisp.place. A static site hosting service built on top of the AT Protocol.
7.9 kB
Markdown
at main

The project is wisp.place. It is a static site hoster built on top of the AT Protocol. The overall basis of the project is that users upload site assets to their PDS as blobs, and creates a manifest record listing every blob as well as site name. The hosting service then catches events relating to the site (create, read, upload, delete) and handles them appropriately.

The lexicons look like this:

//place.wisp.fs
interface Main {
  $type: 'place.wisp.fs'
  site: string
  root: Directory
  fileCount?: number
  createdAt: string
}

interface File {
  $type?: 'place.wisp.fs#file'
  type: 'file'
  blob: BlobRef
  encoding?: 'gzip'
  mimeType?: string
  base64?: boolean
}

interface Directory {
  $type?: 'place.wisp.fs#directory'
  type: 'directory'
  entries: Entry[]
}

interface Entry {
  $type?: 'place.wisp.fs#entry'
  name: string
  node: $Typed<File> | $Typed<Directory> | $Typed<Subfs> | { $type: string }
}

interface Subfs {
  $type?: 'place.wisp.fs#subfs'
  type: 'subfs'
  subject: string  // AT-URI pointing to a place.wisp.subfs record 
  flat?: boolean
}

//place.wisp.subfs
interface Main {
  $type: 'place.wisp.subfs'
  root: Directory
  fileCount?: number
  createdAt: string
}

interface File {
  $type?: 'place.wisp.subfs#file'
  type: 'file'
  blob: BlobRef
  encoding?: 'gzip'
  mimeType?: string
  base64?: boolean
}

interface Directory {
  $type?: 'place.wisp.subfs#directory'
  type: 'directory'
  entries: Entry[]
}

interface Entry {
  $type?: 'place.wisp.subfs#entry'
  name: string
  node: $Typed<File> | $Typed<Directory> | $Typed<Subfs> | { $type: string }
}

interface Subfs {
  $type?: 'place.wisp.subfs#subfs'
  type: 'subfs'
  subject: string  // AT-URI pointing to another place.wisp.subfs record
}

//place.wisp.settings
interface Main {
  $type: 'place.wisp.settings'
  directoryListing: boolean
  spaMode?: string
  custom404?: string
  indexFiles?: string[]
  cleanUrls: boolean
  headers?: CustomHeader[]
}

interface CustomHeader {
  $type?: 'place.wisp.settings#customHeader'
  name: string
  value: string
  path?: string  // Optional glob pattern
}

The main differences between place.wisp.fs and place.wisp.subfs:

  • place.wisp.fs has a required site field
  • place.wisp.fs#subfs has an optional flat field that place.wisp.subfs#subfs doesn't have

The project is a monorepo. The package handler it uses is bun. Please when you want to add a package, which is never unless told to, do bun add ..., please do not try to edit package.json yourself.

Typescript Bun Workspace Layout #

Bun workspaces: packages/@wisp/*, apps/main-app, apps/hosting-service, etc

PLEASE USE bun check to type check and biome check --write to lint.

The services are plain Bun + Effect-free TypeScript: explicit async, no framework runtime dependency. Do not introduce Effect without a reason, and keep configuration resolvable as a pure function of an environment object so it stays testable — resolveDatabaseConfiguration(process.env) in apps/main-app/src/lib/db.ts and resolveHostConfig(env) in apps/hosting-service/src/server.ts are the pattern to copy.

Vendored Repositories #

This project keeps external reference repositories under @repos/ (local checkouts, not tracked in git)

  • Use vendored repositories as read-only reference material when working with related libraries
  • Prefer examples and patterns from the vendored source code over generated guesses or web search results
  • Do not edit files under @repos/ unless explicitly asked
  • Do not import from @repos/ - application code should continue importing from normal package dependencies

Tests conventions #

Both bun test and bun test --isolate (what bun run test passes) must pass from the repo root. A bare bun test loads every file into one module registry, so a test must leave no process-global state behind: restore fake timers, process.env keys and globalThis.fetch it replaced, and never call mock.module. Bun keeps a module mock for the rest of the process (mock.restore() does not undo it), so a mock in one file replaces the real module in every file loaded after it. Instead, a module takes its collaborators as arguments (createFileServing(deps), createApp(deps), createHostingDb({ sql, cache }), createAuthRoutes(deps)) and its production export is that factory applied to the real modules. Prefer real implementations in tests (observability, safe-fetch against a local Bun.serve, tiered-storage on a memory tier) and fake only external services (postgres, redis, a remote PDS) at the injected boundary. bunfig.toml sets pathIgnorePatterns = ["repos/**"] so discovery never reaches the vendored checkouts, whose suites fail to load outside their own dependency tree.

bunfig.toml does register a [test] preload (scripts/test-env.ts), which clears the host, origin and storage variables Bun loads from your local .env before any test module is imported. Without it a developer's .env decides the result: a PRIVATE_HOST changed a 404's reason, and a REDIS_URL turned /health from 200 into a 503 — both failing locally and passing in CI for reasons unrelated to the code. If a test needs one of those variables, set it itself at the top of the file, where the dependency is visible. Run with WISP_TEST_ENV_REPORT=1 to see what was cleared.

The same reasoning applies to any value a module reads at import time: a module-level const X = process.env.X is fixed for the life of the test file and cannot be varied between tests. Prefer passing an environment object to a resolver (resolveHostConfig(env), resolveDatabaseConfiguration(env)) so config stays testable.

There are three typescript apps apps/main-app - Main backend (Bun + Elysia)

  • OAuth authentication and session management
  • Site CRUD operations via PDS
  • Custom domain management
  • React frontend in public/

apps/hosting-service - CDN static file server (Bun + Hono)

  • Serves sites at https://sites.wisp.place/{did}/{site-name} and custom domains
  • Handles redirects (_redirects file support) and routing logic
  • Pulls sites from Tiered Storage (packages/@wispplace/tiered-storage)

apps/firehose-service - ATProto Firehose consumer (Bun or Node)

  • Watches AT Protocol firehose for place.wisp.* record changes
  • Downloads and caches site files to S3
  • Backfill mode for syncing existing sites

apps/webhook-service - ATProto Webhooks service

  • Watches AT Protocol firehose for place.wisp.v2.wh record changes and CRUDs webhooks
  • Watches AT Protocol firehose for scoped aturis based on what is in place.wisp.v2.wh and POSTs them

Shared Packages (packages/@wisp/*) #

  • lexicons - AT Protocol lexicons (place.wisp.fs, place.wisp.subfs, place.wisp.settings) with generated TypeScript types
  • fs-utils - Filesystem tree building, manifest creation, subfs splitting logic
  • atproto-utils - AT Protocol helpers (blob upload, record operations, CID handling)
  • database - PostgreSQL schema and queries
  • constants - Shared constants (limits, file patterns, default settings)
  • observability - OpenTelemetry instrumentation
  • safe-fetch - Wrapped fetch with timeout/retry logic
  • tiered-storage - KV caching where reads bubble up from cold tier to warm/hot tier and writes bubble down from selected tier down. Streaming as well as buffering support. Used to store files in S3 cold tier as source of truth

CLI #

cli-rs/ - Rust workspace (wispctl, published to npm), using jacquard for the AT Protocol.

  • Direct PDS uploads without interacting with main-app
  • Replaces the old TypeScript cli/, which no longer exists. cli-rs/DESIGN.md is the contract for behaviour, wording and on-disk state; the TS CLI was its reference implementation.

Other Directories #

  • docs/ - Astro documentation site
  • binaries/ - Compiled CLI binaries for distribution