diff --git a/.changeset/config.json b/.changeset/config.json index 237216e..745d87f 100644 --- a/.changeset/config.json +++ b/.changeset/config.json @@ -3,7 +3,7 @@ "changelog": "@changesets/cli/changelog", "commit": false, "fixed": [], - "linked": [["@atmo-dev/contrail", "@atmo-dev/contrail-sync"]], + "linked": [["@atmo-dev/contrail", "@atmo-dev/contrail-sync", "@atmo-dev/contrail-community"]], "access": "public", "baseBranch": "main", "updateInternalDependencies": "patch", diff --git a/.changeset/spaces-host-authority-split.md b/.changeset/spaces-host-authority-split.md new file mode 100644 index 0000000..e14e46c --- /dev/null +++ b/.changeset/spaces-host-authority-split.md @@ -0,0 +1,101 @@ +--- +"@atmo-dev/contrail": minor +"@atmo-dev/contrail-community": minor +"@atmo-dev/contrail-sync": minor +--- + +Spaces refactor: split authority + record host into independently runnable +roles, add space credentials, extract community into its own package. + +**Breaking — config shape** + +`spaces` is no longer flat — split into `authority` and `recordHost`: + +```ts +// before +spaces: { + type: "com.example.event.space", + serviceDid: "did:web:example.com", + blobs: { adapter, maxSize }, +} + +// after +spaces: { + authority: { + type: "com.example.event.space", + serviceDid: "did:web:example.com", + signing: await generateAuthoritySigningKey(), + }, + recordHost: { + blobs: { adapter, maxSize }, + }, +} +``` + +**Breaking — community moved to its own package** + +Community has been extracted to `@atmo-dev/contrail-community`. Wire it via +`createCommunityIntegration`: + +```ts +import { Contrail, resolveConfig } from "@atmo-dev/contrail"; +import { createCommunityIntegration } from "@atmo-dev/contrail-community"; + +const resolved = resolveConfig(config); +const communityIntegration = createCommunityIntegration({ db, config: resolved }); +const contrail = new Contrail({ ...config, communityIntegration }); +``` + +The community config (`config.community`) stays the same; only the wiring +moves. Imports of `CommunityAdapter`, `registerCommunityRoutes`, +`reconcile`, etc. now come from `@atmo-dev/contrail-community` instead of +`@atmo-dev/contrail`. + +**New — space credentials (`X-Space-Credential`)** + +The space authority issues short-lived ES256 JWTs (default 2h TTL) via +`.space.getCredential` and `refreshCredential`. The record host accepts +them on read/write paths in lieu of per-request service-auth JWTs. Skips +DID-doc fetches and member checks; the credential's signature is the proof. + +Generate a signing key once at deploy time: + +```ts +import { generateAuthoritySigningKey } from "@atmo-dev/contrail"; +const signing = await generateAuthoritySigningKey(); +// Store the JWK; pass to spaces.authority.signing. +``` + +**New — binding resolution** + +Verifiers can resolve "which authority signs for this space?" from three +sources, in order: local enrollment table, PDS records at +`at:////`, DID-doc `#atproto_space_authority` service +entry, owner-self fallback. Lets user-owned DIDs authorize a third-party +authority via a normal PDS write — no DID-doc surgery. + +**New — independent deployments + enrollment** + +The authority and record host can run as separate processes/operators. +A new `.recordHost.enroll` endpoint lets owners (or authorities) +register a space onto a host. In-process deployments auto-enroll on +`createSpace`; nothing changes for single-instance setups. + +See `docs/10-deployment-shapes.md` for all-in-one / authority-only / +host-only configurations and when to choose each. + +**Migration** + +For most deployments running spaces today, the migration is: + +1. Update the config: split `spaces.{type, serviceDid, blobs}` into + `spaces.authority.{type, serviceDid}` and `spaces.recordHost.{blobs}`. +2. Generate and store an authority signing key + (`generateAuthoritySigningKey()`); add to `spaces.authority.signing`. +3. If using community: install `@atmo-dev/contrail-community`, build + `createCommunityIntegration({ db, config })`, pass via + `new Contrail({ communityIntegration })` (or `createApp({ community })`). + +Existing service-auth JWT clients keep working as a fallback path. +Migrate to space credentials when convenient — exchange a JWT for a +credential once via `getCredential`, then reuse it. diff --git a/README.md b/README.md index 1f88ec9..c4203e0 100644 --- a/README.md +++ b/README.md @@ -89,13 +89,15 @@ returns every `community.lexicon.calendar.event` record published anywhere on at - [Communities](https://github.com/flo-bit/contrail/blob/main/docs/07-communities.md) — group-controlled atproto DIDs - [Sync](https://github.com/flo-bit/contrail/blob/main/docs/08-sync.md) — reactive client-side store over `watchRecords` - [Labels](https://github.com/flo-bit/contrail/blob/main/docs/09-labels.md) — atproto-native moderation hydration from external labelers +- [Deployment shapes](https://github.com/flo-bit/contrail/blob/main/docs/10-deployment-shapes.md) — all-in-one vs split-authority vs split-host configurations - Frameworks: [SvelteKit + Cloudflare](https://github.com/flo-bit/contrail/blob/main/docs/frameworks/sveltekit-cloudflare.md) ## Packages | Package | | |---|---| -| `@atmo-dev/contrail` | Core library — indexing, XRPC server, spaces, communities, realtime | +| `@atmo-dev/contrail` | Core library — indexing, XRPC server, spaces, realtime | +| `@atmo-dev/contrail-community` | Community module — group-controlled DIDs, access-level ladder. Plugs into core via an integration | | `@atmo-dev/contrail-sync` | Client-side reactive watch-store with optional IndexedDB cache | | `@atmo-dev/contrail-lexicons` | Codegen + `contrail-lex` CLI | diff --git a/docs/05-auth.md b/docs/05-auth.md index ad8f659..7cb7578 100644 --- a/docs/05-auth.md +++ b/docs/05-auth.md @@ -1,15 +1,18 @@ # Auth -Contrail has four auth mechanisms. Which one applies depends on who's calling and what they're asking for. +Contrail has six auth mechanisms. Which one applies depends on who's calling, where they're calling, and what they're asking for. | Mechanism | Used by | For | |---|---|---| | Anonymous | anyone | public reads | -| Service-auth JWT | third-party apps acting on behalf of a user | anything permissioned (spaces, communities) | +| Service-auth JWT | third-party apps acting on behalf of a user | authority-side ops + record-host fallback | +| **Space credential** (`X-Space-Credential`) | callers after exchange via `getCredential` | record-host reads/writes — primary path | | In-process server client | your own server code | loaders / actions that skip HTTP entirely | | Invite token | anonymous bearers | read-only access to a specific space | | Watch ticket | browsers | realtime subscriptions (`watchRecords`) | +The two "atproto-y" mechanisms (service-auth JWTs and space credentials) work in tandem on permissioned routes: a caller exchanges a JWT for a credential once via `space.getCredential`, then presents the credential on every subsequent record-host request until it expires. + ## Service-auth JWTs The standard atproto mechanism. When a third-party app wants to call your contrail service as a user, it: @@ -20,23 +23,87 @@ The standard atproto mechanism. When a third-party app wants to call your contra Contrail verifies every request against the public key in the issuer's DID doc (`@atcute/xrpc-server` does the heavy lifting). It checks: - Signature valid -- `aud` matches the `serviceDid` you configured +- `aud` matches the `serviceDid` you configured (under `spaces.authority.serviceDid`) - `lxm` covers the method being called - Token hasn't expired On pass, your handler sees a populated `serviceAuth = { issuer, audience, lxm, clientId }` context and can proceed. On fail, 401 or 403 with a structured reason. +### Where service-auth JWTs apply + +- **Authority routes** (`.space.createSpace`, `addMember`, `getCredential`, etc.) — JWT-only. Credentials are scoped to record-host operations; you can't use one to manage spaces. +- **Record-host routes** (`putRecord`, `listRecords`, `uploadBlob`, etc.) — accept JWTs as a fallback path. The credential path (below) is preferred. + ### The `serviceDid` gotcha Use the **plain DID** (no `#fragment`) when configuring contrail: ```ts -spaces: { serviceDid: "did:web:example.com" } // right -spaces: { serviceDid: "did:web:example.com#com_example_space" } // wrong +spaces: { authority: { serviceDid: "did:web:example.com" } } // right +spaces: { authority: { serviceDid: "did:web:example.com#com_example_x" } } // wrong ``` Many PDS implementations reject `aud` values containing `#fragment` in `com.atproto.server.getServiceAuth`, and contrail does strict string equality on `aud`. The fragment form belongs only in your DID doc's `service` entry, where PDSes use it to resolve the service endpoint URL for `Atproto-Proxy` routing — that's separate from JWT audience validation. +## Space credentials + +Short-lived (default 2h) ES256 JWTs minted by the space authority. Once a caller has one, they present it via `X-Space-Credential: ` on every record-host request and skip the per-request JWT mint dance. This matches the rough atproto permissioned-data spec. + +### Lifecycle + +``` +1. Caller mints a service-auth JWT { aud, lxm: ".space.getCredential" }. +2. POST .space.getCredential { spaceUri } Authorization: Bearer + → { credential: "", expiresAt: } +3. Caller stores the credential. For ~2 hours, every record-host request: + X-Space-Credential: + succeeds without going back through the user's PDS. +4. Before expiry, refresh: + POST .space.refreshCredential { credential } + → { credential: , expiresAt: } +``` + +### Claims + +```json +{ + "iss": "", + "sub": "", + "space": "ats:////", + "scope": "rw", + "iat": 1746000000, + "exp": 1746014400 +} +``` + +- Signed with the authority's ES256 key (kid = `#atproto_space_authority`). +- Stateless — verifiable by anyone who can resolve the authority DID's verification key. +- `scope` is `"rw"` or `"read"`. Today only `rw` is issued via `getCredential`; the read-only path is a future read-grant invite replacement. + +### How verification works + +When the record host receives a credential, it: + +1. Decodes the JWT, reads `iss` and `space`. +2. Asks its **binding resolver** "who's authorized to sign for this space?" — primary source is the local enrollment table; fallbacks include PDS records and DID-doc service entries. +3. Confirms `iss` matches the authorized DID. +4. Resolves the issuer's verification key (local in-process, or via DID doc). +5. Verifies signature, expiry, scope, space match. + +In an in-process deployment, this is one DB lookup + one signature verification. No DID-doc fetches per request. See [Spaces](./06-spaces.md#discovery--binding-resolution) for the binding details. + +### When the credential gets rejected + +| Reason | Response | +|---|---| +| `malformed` | 401 — JWT structure invalid | +| `bad-alg` | 401 — header alg ≠ ES256 | +| `bad-signature` | 401 — signature didn't verify against the resolved key | +| `expired` | 401 — past `exp`. Refresh, or re-mint. | +| `wrong-space` | 403 — credential's `space` ≠ request's space | +| `wrong-scope` | 403 — read-only credential on a write | +| `unknown-issuer` | 401 — `iss` doesn't match the binding for the space (most often: not enrolled here) | + ## In-process server client When your own server code wants to call contrail, the service-auth dance is pointless — it's your code talking to your code. `createServerClient` skips it: @@ -50,7 +117,7 @@ const client = createServerClient(async (req) => handle(req, env.DB), userDid); const res = await client.get("com.example.event.listRecords", { params: {...} }); ``` -Pass `did` to act as that user; omit it for anonymous calls against public endpoints. This is a trust boundary — anything that actually crosses a network needs a real service-auth JWT, not this shortcut. +Pass `did` to act as that user; omit it for anonymous calls against public endpoints. This is a trust boundary — anything that actually crosses a network needs a real service-auth JWT or space credential, not this shortcut. See [SvelteKit + Cloudflare](./frameworks/sveltekit-cloudflare.md) for the typical loader pattern. @@ -59,7 +126,7 @@ See [SvelteKit + Cloudflare](./frameworks/sveltekit-cloudflare.md) for the typic First-class auth for spaces. When a space owner creates an invite: ``` -com.example.space.invite.create { spaceUri, ttl?, maxUses? } +.invite.create { spaceUri, kind, ttl?, maxUses? } → { token: "...plaintext..." } // returned once, never again ``` @@ -67,7 +134,7 @@ The plaintext token is handed to the user out-of-band (link, QR, email). Contrai Three invite kinds, depending on what the token does: -- **`join`** — redeemed via `com.example.space.invite.redeem` with a service-auth JWT. Adds the caller's DID to the member list. Members have full read + write inside the space; there's no per-member permission axis beyond "is a member." +- **`join`** — redeemed via `.invite.redeem` with a service-auth JWT. Adds the caller's DID to the member list. Members have full read + write inside the space; there's no per-member permission axis beyond "is a member." - **`read`** — bearer-only. The token itself grants read access when passed as `?inviteToken=`, no DID, no redemption. Good for sharing a read-only link that doesn't add anyone to the member list. - **`read-join`** — both. Works anonymously as a read token; can also be redeemed with a JWT to promote the caller to member. @@ -79,7 +146,7 @@ Realtime subscriptions (`watchRecords`) can't use regular service-auth JWTs for Server-side minting comes in two flavours: -- `com.example.realtime.ticket` — POST `{ topic }` (e.g. `"space:ats://..."`) → `{ ticket, topics, expiresAt }`. Bare topic-list ticket, used with the generic `<ns>.realtime.subscribe` endpoint. +- `<ns>.realtime.ticket` — POST `{ topic }` (e.g. `"space:ats://..."`) → `{ ticket, topics, expiresAt }`. Bare topic-list ticket, used with the generic `<ns>.realtime.subscribe` endpoint. - `<collection>.watchRecords?mode=ws&spaceUri=…` (or `&actor=…`) handshake — returns `{ snapshot, ticket, wsUrl, sinceTs, ticketTtlMs, querySpec }`. The ticket is bound to `(did, topics, querySpec)` and is the one to use for the per-collection `watchRecords` stream — both for SSE (`?ticket=…`) and the subsequent WS upgrade. Both flavours are signed by `realtime.ticketSecret` (a 32-byte random, configured once). Clients hand the ticket off via `?ticket=...` on connect. @@ -129,9 +196,14 @@ A typical flow for a third-party app acting as a user in a space: 1. App registers OAuth client pointing at your permission set NSID. 2. User grants consent — PDS fetches your permission set lexicon via DNS, shows the user the methods, records the scope. -3. App calls `com.atproto.server.getServiceAuth` on the user's PDS: `{ aud: "did:web:example.com", lxm: "com.example.space.putRecord", exp: <60s> }`. PDS signs, returns JWT. -4. App sends `PUT /xrpc/com.example.space.putRecord` with `Authorization: Bearer <jwt>` and `Atproto-Proxy: did:web:example.com#com_example_space`. -5. User's PDS reads the `Atproto-Proxy` header, resolves the service endpoint from your DID doc, forwards the request. -6. Contrail verifies the JWT (signature, `aud`, `lxm`, expiry), runs ACL check (is this DID a member of that space?), dispatches the write. - -For your own loaders/actions, steps 3–6 collapse into a single `createServerClient({did}).post(...)` call. For a browser subscribing to a feed, steps 3–5 are replaced by a ticket mint from your server. Same auth model, different surface. +3. App calls `com.atproto.server.getServiceAuth` on the user's PDS: `{ aud: "did:web:example.com", lxm: "com.example.space.getCredential", exp: <60s> }`. PDS signs, returns JWT. +4. App POSTs `<ns>.space.getCredential { spaceUri }` with `Authorization: Bearer <jwt>` and `Atproto-Proxy: did:web:example.com#com_example_space`. Authority verifies the JWT, checks membership, mints a 2h credential. +5. App caches the credential. Every subsequent `putRecord` / `listRecords` / `uploadBlob`: + ``` + POST <ns>.space.putRecord + X-Space-Credential: <credential> + ``` + The record host verifies the credential against its enrolled authority's key — no PDS roundtrip, no DID-doc fetch. +6. Before expiry, app calls `refreshCredential` to get a fresh one. + +For your own loaders/actions, the credential dance disappears — `createServerClient({did}).post(...)` bypasses it entirely. For a browser subscribing to a feed, steps 3–5 are replaced by a ticket mint from your server. Same auth model, different surface. diff --git a/docs/06-spaces.md b/docs/06-spaces.md index 8bb9fb8..338abc7 100644 --- a/docs/06-spaces.md +++ b/docs/06-spaces.md @@ -12,17 +12,40 @@ Auth-gated store for records that can't live on public PDSes — private events, Every permission boundary is its own space. No nested ACLs. Richer roles = more spaces or app-layer checks. +### Two roles, one or two services + +A space has two operational roles: + +- **Space authority** — owns the member list, signs short-lived credentials. Identified by a service DID. +- **Record host** — stores records and blobs for spaces it has *enrolled*. + +In the default deployment both run in the same Contrail instance against the same DB; you don't notice the split. But the roles can also run separately — see [deployment shapes](./10-deployment-shapes.md) for ACL-on-arbiter / records-on-contrail patterns. + ## Enable ```ts import type { ContrailConfig } from "@atmo-dev/contrail"; +import { generateAuthoritySigningKey } from "@atmo-dev/contrail"; + +// One-time setup: generate a signing key and store it. The authority signs +// space credentials with this key; verifiers find the public key in the +// authority DID's DID document or via the binding-resolver chain. +const signing = await generateAuthoritySigningKey(); const config: ContrailConfig = { namespace: "com.example", collections: { /* ... */ }, spaces: { - type: "com.example.event.space", - serviceDid: "did:web:example.com", + authority: { + type: "com.example.event.space", + serviceDid: "did:web:example.com", + signing, // omit to disable credential issuance + credentialTtlMs: 2 * 60 * 60 * 1000, // 2h, matches the rough spec + }, + recordHost: { + // blobs is optional; omit to disable blob endpoints + blobs: { adapter: blobsAdapter }, + }, }, }; ``` @@ -33,42 +56,98 @@ Each collection gets a parallel `spaces_records_<short>` table. Opt out per-coll public_only: { collection: "com.example.public", allowInSpaces: false } ``` -## Auth +## Auth — three paths + +The record host accepts three forms of auth on read/write paths, in this precedence order: -Spaces use the standard contrail auth surface — service-auth JWTs for third-party apps, in-process server clients for your own loaders, invite tokens for anonymous read-grant links. See [Auth](./05-auth.md) for the full picture. +1. **`X-Space-Credential` header** — a short-lived JWT minted by the space authority. The primary path: callers exchange a service-auth JWT once via `space.getCredential`, then present the credential on every request until it expires. Skips per-request DID-doc fetches and member checks; the credential's signature is the proof. +2. **`?inviteToken=...` query** (read-only) — bearer access for shareable links. See [Auth § Invite tokens](./05-auth.md#invite-tokens). +3. **`Authorization: Bearer <service-auth-jwt>`** — the standard atproto path. Useful for one-off calls (the credential exchange itself, space-management endpoints) or as a fallback when the caller doesn't want to manage credentials. + +Authority-side endpoints (`createSpace`, `addMember`, `getCredential`, etc.) only accept service-auth JWTs — credentials are scoped to record-host operations. + +See [Auth](./05-auth.md) for the full picture. + +## Credential flow + +```text + ┌──────────────┐ + │ user PDS │ mints service-auth JWT (lxm: getCredential) + └──────┬───────┘ + ▼ + ┌──────────────┐ + │ authority │ validates JWT, checks membership, + │ (Contrail) │ signs ES256 credential (2h TTL) + └──────┬───────┘ + │ { credential, expiresAt } + ▼ + ┌──────────────┐ + │ record host │ verifies credential signature against + │ (Contrail or │ authority DID's published key, + │ elsewhere) │ checks scope/space/expiry, serves request + └──────────────┘ +``` -Space-specific wiring: +`space.refreshCredential` re-issues a fresh credential from an unexpired one without going back through the JWT mint dance — useful for long-running clients. -- `serviceDid` in the config is the `aud` contrail expects on incoming JWTs. Plain DID, no `#fragment`. -- Apps acting in a space send `Atproto-Proxy: <serviceDid>#<service-id-from-your-did-doc>` so the user's PDS routes correctly. -- Invite redemption via service-auth JWT grants membership; via `?inviteToken=...` query param grants read-only bearer access to that space. +## Enrollment + +The record host maintains a local table of which spaces it accepts records for and which authority signs credentials for each. Two ways enrollment happens: + +- **Auto-enroll** (default for in-process deployments): the authority's `createSpace` automatically enrolls the new space on the colocated record host. New users see no enrollment surface; it just works. +- **Explicit `recordHost.enroll`**: for split deployments where the authority and record host run in different processes/operators, the owner (or the authority itself) calls `<ns>.recordHost.enroll { spaceUri, authority }` to consent. Idempotent — re-enrolling updates the binding. + +A non-enrolled space gets 404 "not-enrolled" on every record-host route. This is the host's consent layer — without it, anyone with a valid credential could create unbounded storage on your host. + +## Discovery — binding resolution + +When a record host receives a credential, it needs to know whether the credential's `iss` is authorized to sign for that space. Three sources, tried in order: + +1. **Local enrollment** — primary on the record host. `(spaceUri → authorityDid)` from the enrollment table. +2. **PDS record** at `at://<owner>/<type>/<key>` — for user-owned DIDs that declared a host via a normal PDS write. Lexicon: `tools.atmo.space.declaration` (or your namespaced variant). +3. **DID-doc service entry** — `#atproto_space_authority` on the owner's DID doc. For provisioned (no-PDS) DIDs. +4. **Owner self-issues** (fallback) — for the trivial case where the owner DID's own key signs credentials. + +For in-process deployments, step 1 is the only one that fires. The other resolvers are wired in by deployments that accept credentials from external authorities — see [deployment shapes](./10-deployment-shapes.md). ## Unified `listRecords` | Call | Returns | |---|---| | no auth, no `spaceUri` | public only | -| `?spaceUri=…` + JWT | one space (ACL-gated) | -| JWT, no `spaceUri` | public **unioned** with every space the caller is a member of | +| `?spaceUri=…` + credential or JWT | one space (ACL-gated) | +| credential / JWT, no `spaceUri` | public **unioned** with every space the caller is a member of | Filters, sorts, hydration, and references work across all three. Records from a space carry a `space: <spaceUri>` field — same on `listRecords`/`getRecord` responses and `watchRecords` stream events. ## Invites -First-class primitive — see [Auth § Invite tokens](./05-auth.md#invite-tokens) for the mechanism. Space-specific: create via `com.example.space.invite.create`, redeem via `.redeem` (membership grant) or `?inviteToken=...` query param (read-only bearer grant). +First-class primitive — see [Auth § Invite tokens](./05-auth.md#invite-tokens) for the mechanism. Space-specific: create via `<ns>.invite.create`, redeem via `.redeem` (membership grant) or `?inviteToken=...` query param (read-only bearer grant). ## XRPCs -- `com.example.space.create | get | list | delete` -- `com.example.space.putRecord | deleteRecord | listRecords | getRecord` -- `com.example.space.invite.create | redeem | revoke | list` -- `com.example.space.listMembers | removeMember` +### Authority routes (`<ns>.space.*` — spec-aligned) +- `createSpace` `getSpace` `listSpaces` `deleteSpace` +- `listMembers` `addMember` `removeMember` +- `getCredential` `refreshCredential` +- `leaveSpace` (contrail extra) + +### Record-host routes (`<ns>.space.*` for records, `<ns>.recordHost.*` for management) +- `putRecord` `deleteRecord` `getRecord` `listRecords` +- `uploadBlob` `getBlob` `listBlobs` (when `recordHost.blobs` is configured) +- `recordHost.enroll` + +### Contrail extras (`<ns>.spaceExt.*`) +- `whoami` — caller's relationship to a space (extensions plug in via the integration's whoami hook) + +### Invite (`<ns>.invite.*`) +- `create` `redeem` `revoke` `list` ## What's not here - No E2EE (data is operator-readable). - No FTS on `?spaceUri=…` yet. -- No per-space sharding — one DB, one operator. -- Not a long-term replacement for real atproto permissioned repos. +- Records still live in the operator's DB rather than user PDSes — federation is greenfield. (See `refs/spaces-spec-mapping.md` for the migration notes.) +- No managing-app routing (join requests, approval queues — see `refs/spaces-later.md`). -The design follows Daniel Holmgren's [permissioned data rough spec](https://dholms.leaflet.pub/3mhj6bcqats2o). The goal is that when real atproto permissioned repos ship, migration is mostly data movement — the API your app speaks doesn't change. +The design follows Daniel Holmgren's [permissioned data rough spec](https://dholms.leaflet.pub/3mhj6bcqats2o). When real atproto permissioned repos ship, migration is mostly data movement — the wire surface your app speaks doesn't change. diff --git a/docs/07-communities.md b/docs/07-communities.md index a7f2e26..0268c42 100644 --- a/docs/07-communities.md +++ b/docs/07-communities.md @@ -2,6 +2,56 @@ Group-controlled atproto DIDs. A community is a DID whose signing/rotation keys are held by the appview on behalf of multiple members, with tiered access levels. Built on top of [spaces](./06-spaces.md). +Communities live in a separate package — `@atmo-dev/contrail-community` — that plugs into Contrail via an integration object. The contrail core has no knowledge of community-specific concepts; the package wires itself in via injectable hooks (whoami extension, invite handler, route registration, schema). + +## Install + +```bash +pnpm add @atmo-dev/contrail @atmo-dev/contrail-community +``` + +## Wire it up + +Construct the integration once, hand it to `Contrail` (or directly to `createApp`): + +```ts +import { Contrail, resolveConfig, type ContrailConfig } from "@atmo-dev/contrail"; +import { createCommunityIntegration } from "@atmo-dev/contrail-community"; + +const config: ContrailConfig = { + namespace: "com.example", + collections: { /* ... */ }, + spaces: { + authority: { type: "com.example.event.space", serviceDid: "did:web:example.com", signing }, + recordHost: {}, + }, + community: { + masterKey: env.COMMUNITY_MASTER_KEY, // 32-byte encryption key for stored credentials + serviceDid: "did:web:example.com", + levels: ["admin", "moderator"], // ranked, highest-first + }, +}; + +const resolved = resolveConfig(config); +const communityIntegration = createCommunityIntegration({ db, config: resolved }); + +const contrail = new Contrail({ ...config, db, communityIntegration }); +await contrail.init(); // applies community schema alongside contrail's own +``` + +Or with `createApp` directly: + +```ts +import { createApp, initSchema } from "@atmo-dev/contrail"; +import { createCommunityIntegration } from "@atmo-dev/contrail-community"; + +const community = createCommunityIntegration({ db, config }); +await initSchema(db, config, { extraSchemas: [community.applySchema] }); +const app = createApp(db, config, { community }); +``` + +Stored credentials (app passwords for adopted communities, signing keys for minted) are envelope-encrypted with `masterKey`. Never ship the placeholder. + ## When to use this When you want atproto records published under a *shared* identity — a team, a project, a channel — not a single user. Think: a group's published calendar events, a community's published posts. @@ -15,17 +65,7 @@ Either way, the result is the same: a DID that multiple members can act through, ## Access levels -Each member has a level (ranked). Levels map to write permissions. Owners can grant/revoke levels. Two reserved levels exist: `owner` and `member`. Your deployment defines the rest: - -```ts -community: { - masterKey: ENV.COMMUNITY_MASTER_KEY, // 32-byte encryption key for stored credentials - serviceDid: "did:web:example.com", - levels: ["admin", "moderator"], // ranked, highest-first -} -``` - -Stored credentials (app passwords for adopted communities, signing keys for minted) are envelope-encrypted with `masterKey`. Never ship the placeholder. +Each member has a level (ranked). Levels map to write permissions. Owners can grant/revoke levels. Two reserved levels exist: `owner` and `member`. Your deployment defines the rest via `config.community.levels`. ## How it composes with spaces @@ -37,13 +77,18 @@ community.space.grant { spaceUri, subject: { did: "did:plc:..." }, accessLevel: The spaces layer stays ignorant of access levels — it just sees "this DID is a member." The community layer projects member × level → membership in specific spaces. Once a DID is a member of a space (through a community grant or otherwise), they have full read + write inside it. +The integration plugs in to two contrail extension points: + +- **Whoami** — `<ns>.spaceExt.whoami` returns `accessLevel` for community-owned spaces (the community whoami extension overrides the default binary-membership response). +- **Invites** — the unified `<ns>.invite.*` family dispatches community-owned spaces through the community invite handler (which uses access levels) and user-owned spaces through the spaces module's binary-membership handler. + ## XRPCs -- `com.example.community.mint | adopt | list | delete` -- `com.example.community.invite.create | redeem | revoke | list` -- `com.example.community.setAccessLevel | revoke | listMembers` -- `com.example.community.space.create | grant | revoke | ...` — community-owned spaces -- `com.example.community.putRecord | deleteRecord` — publish records as the community DID +- `<ns>.community.mint | adopt | list | delete` +- `<ns>.community.invite.create | redeem | revoke | list` +- `<ns>.community.setAccessLevel | revoke | listMembers` +- `<ns>.community.space.create | grant | revoke | ...` — community-owned spaces +- `<ns>.community.putRecord | deleteRecord` — publish records as the community DID ## What's not here diff --git a/docs/10-deployment-shapes.md b/docs/10-deployment-shapes.md new file mode 100644 index 0000000..25464ed --- /dev/null +++ b/docs/10-deployment-shapes.md @@ -0,0 +1,199 @@ +# Deployment shapes + +Spaces split into two roles, run together by default. Three deployment shapes, in increasing order of complexity: + +1. **All-in-one** — authority + record host + (optional) community in one process. The default; what you get from `createApp` with both `spaces.authority` and `spaces.recordHost` configured. Most apps want this. +2. **Authority-only** — a service that controls ACL and signs credentials, but doesn't store records. Useful when records live on someone else's host (e.g. a community arbiter that delegates storage to a heavier appview). +3. **Record-host-only** — a service that stores records, accepting credentials signed by an external authority. Useful when storage lives separately from governance — e.g. Contrail-as-host backing spaces that an "Arbiter" or HappyView manages. + +The split is real at the wire level (different XRPCs, different auth shapes) but the same Contrail codebase handles all three. This doc walks through each. + +## Shape 1: all-in-one (default) + +``` +┌────────────────────────────────────┐ +│ Contrail │ +│ ┌────────┐ ┌────────────┐ │ +│ │authority│ │ record host│ │ +│ │ + signing│ │ + enrollment│ │ +│ └────────┘ └────────────┘ │ +│ shared DB, single process │ +└────────────────────────────────────┘ +``` + +Config: + +```ts +spaces: { + authority: { + type: "com.example.event.space", + serviceDid: "did:web:example.com", + signing: await generateAuthoritySigningKey(), + }, + recordHost: { + blobs: { adapter: blobsAdapter }, // optional + }, +}, +``` + +What happens at startup: + +- `initSchema` creates both authority tables (`spaces`, `spaces_members`, `spaces_invites`) and record-host tables (`spaces_records_<short>`, `spaces_blobs`, `record_host_enrollments`). +- The umbrella router wires `registerAuthorityRoutes` + `registerRecordHostRoutes` against the same `HostedAdapter`. +- The credential verifier is built from `Local` binding + `Local` key — no DID-doc fetches; the host knows the authority's public key directly. + +What happens when a user creates a space: + +1. `createSpace` writes a row in `spaces` (authority) and immediately a row in `record_host_enrollments` (host). One round-trip, two DB writes. +2. From there, `getCredential` works, `putRecord` works, the world is in sync. + +This is the path most apps run. You don't notice the role split. + +## Shape 2: authority-only + +A lightweight service that holds ACL and signs credentials. Records live on someone else's host. + +``` + ┌────────────────┐ + │ this Contrail │ + │ authority │ + └────────────────┘ + ▲ + │ getCredential + │ + ┌─────────────┐ + │ client │ + └─────────────┘ + │ X-Space-Credential + ▼ + ┌─────────────┐ + │ external │ enrolled with this authority + │ record host │ (different operator, different DID) + └─────────────┘ +``` + +Config: + +```ts +spaces: { + authority: { + type: "com.example.event.space", + serviceDid: "did:web:authority.example.com", + signing: await generateAuthoritySigningKey(), + }, + // recordHost omitted — this deployment doesn't store records +}, +``` + +`createSpace` here does NOT auto-enroll anywhere. The space owner (or the authority itself) calls `recordHost.enroll` on whichever host they want to use; the host then accepts credentials signed by this authority for that space. + +The authority's DID document needs to publish the verification key under `#atproto_space_authority` so external hosts can resolve it. + +## Shape 3: record-host-only + +A storage tier that accepts credentials signed by external authorities. + +``` +┌────────────┐ +│ external │ signs credentials +│ authority │ +└────────────┘ + │ + ▼ +┌─────────────────────────────────────┐ +│ this Contrail │ +│ record host (no authority) │ +│ │ +│ verifies credentials via: │ +│ - enrollment table │ +│ - DID-doc key resolver │ +│ (for external authorities) │ +└─────────────────────────────────────┘ +``` + +Config: + +```ts +spaces: { + // authority is still needed for the JWT verifier infrastructure (so the + // record host can validate JWTs on the recordHost.enroll endpoint), but + // no signing key is configured — this deployment doesn't issue creds. + authority: { + type: "com.example.event.space", + serviceDid: "did:web:host.example.com", + }, + recordHost: { + blobs: { adapter: blobsAdapter }, + }, +}, +``` + +To accept credentials from an external authority, wire a custom verifier: + +```ts +import { + createApp, + createBindingCredentialVerifier, + createEnrollmentBindingResolver, + createDidDocKeyResolver, +} from "@atmo-dev/contrail"; +import { CompositeDidDocumentResolver, PlcDidDocumentResolver, WebDidDocumentResolver } + from "@atcute/identity-resolver"; + +const didResolver = new CompositeDidDocumentResolver({ + methods: { + plc: new PlcDidDocumentResolver(), + web: new WebDidDocumentResolver(), + }, +}); + +const verifier = createBindingCredentialVerifier({ + // Local enrollment is the canonical binding source — only spaces this + // host has explicitly opted into are accepted. + bindings: createEnrollmentBindingResolver({ recordHost: hostAdapter }), + // For credential signature verification, walk DID docs of external + // authorities to find their published verification keys. + keys: createDidDocKeyResolver({ resolver: didResolver }), +}); + +const app = createApp(db, config, { + spaces: { credentialVerifier: verifier }, +}); +``` + +The flow when a request arrives: + +1. Caller presents `X-Space-Credential: <jwt>`. +2. Verifier reads `iss` from the JWT, looks up enrollment for `claims.space`. If the enrollment's `authorityDid` matches `iss` → continue. If not → 401 `unknown-issuer`. +3. Resolves the issuer DID, finds the verification method with id matching the JWT's `kid`, verifies the signature. +4. Checks expiry, scope, space match. +5. Serves the request. + +Enrollment is the host's consent layer: a credential can only be presented for spaces the host has agreed to store. Without enrollment, no records get written. + +## Mixing shapes + +You can run all three simultaneously in one Contrail instance. The umbrella router enables each set of routes based on what's configured: + +- `spaces.authority` → authority routes registered (`createSpace`, `getCredential`, etc.) +- `spaces.recordHost` → record-host routes registered (`putRecord`, `recordHost.enroll`, etc.) +- Both → today's default. + +A deployment can act as the authority for spaces it owns *and* a record host for spaces other authorities own. Auto-enroll fires only for spaces this deployment is the authority for; external authorities still enroll explicitly. + +## Choosing a shape + +| Need | Shape | +|---|---| +| One operator, one process, want it to work | All-in-one | +| You're running a "DAO governance / arbiter" service that decides ACL but not storage | Authority-only | +| You're running an appview / heavier storage tier and want to accept ACL decisions from external services | Record-host-only | +| You're an existing Contrail deployment that wants to also accept external authorities | All-in-one + custom verifier | + +When in doubt, all-in-one. Splitting is for when you have a real operational reason to separate the two — different teams running them, different latency profiles, different scaling targets, different governance. + +## What's not here + +- **Authority migration** — moving a space's authority from DID A to DID B. The architecture supports it (re-enroll on the host with the new authority binding) but no helper API yet. +- **Multi-authority per space** — could in principle allow several authorities to all sign for one space (replication scenarios). Not modeled today; spec is silent. +- **PDS-backed records** — when atproto's permissioned-repos protocol ships, records will federate from user PDSes. The host becomes an aggregator rather than a store. The role split here generalizes to that world without changes. diff --git a/packages/contrail-community/package.json b/packages/contrail-community/package.json index 2fae478..43fe2e1 100644 --- a/packages/contrail-community/package.json +++ b/packages/contrail-community/package.json @@ -1,6 +1,6 @@ { "name": "@atmo-dev/contrail-community", - "version": "0.1.0", + "version": "0.4.2", "description": "Community module for contrail — community-owned spaces with tiered access levels (member → moderator → admin), invite tokens, DID provisioning, and the access-level reconciler that keeps spaces_members in sync.", "type": "module", "sideEffects": false, diff --git a/packages/contrail-community/tests/realtime-community.test.ts b/packages/contrail-community/tests/realtime-community.test.ts new file mode 100644 index 0000000..c6c6b7a --- /dev/null +++ b/packages/contrail-community/tests/realtime-community.test.ts @@ -0,0 +1,148 @@ +/** Realtime + community integration test. Lives here (not in contrail) so + * contrail's package.json doesn't have to dev-depend on contrail-community + * (which would create a build-graph cycle in turbo). + * + * Tests that `community:<did>` topics expand to the caller's reachable + * community spaces — the cross-cutting concern that needs both modules. */ + +import { describe, it, expect, beforeAll } from "vitest"; +import { Hono } from "hono"; +import type { MiddlewareHandler } from "hono"; +import { createSqliteDatabase } from "@atmo-dev/contrail/sqlite"; +import { + createApp, + initSchema, + resolveConfig, + type ContrailConfig, +} from "@atmo-dev/contrail"; +import { createCommunityIntegration } from "../src/integration"; + +const ALICE = "did:plc:alice"; +const CHARLIE = "did:plc:charlie"; + +const MASTER_KEY = new Uint8Array(32).fill(5); +const REALTIME_SECRET = new Uint8Array(32).fill(9); + +const CONFIG: ContrailConfig = { + namespace: "test.rt", + collections: { message: { collection: "app.event.message" } }, + spaces: { + authority: { + type: "tools.atmo.event.space", + serviceDid: "did:web:test.example#svc", + }, + recordHost: {}, + }, + community: { + masterKey: MASTER_KEY, + plcDirectory: "https://plc.test", + fetch: mockFetch, + resolver: mockResolver(), + }, + realtime: { + ticketSecret: REALTIME_SECRET, + keepaliveMs: 60_000, + }, +}; + +function mockResolver(): any { + return { + resolve: async (_did: string) => ({ + id: _did, + service: [ + { id: "#atproto_pds", type: "AtprotoPersonalDataServer", serviceEndpoint: "https://pds.test" }, + ], + }), + }; +} + +async function mockFetch(input: RequestInfo | URL, init?: RequestInit): Promise<Response> { + const url = typeof input === "string" ? input : input instanceof URL ? input.href : input.url; + if (url.endsWith("/xrpc/com.atproto.server.createSession")) { + return new Response( + JSON.stringify({ accessJwt: "a.b.c", refreshJwt: "r.r.r", did: "did:plc:community" }), + { status: 200, headers: { "content-type": "application/json" } } + ); + } + if (url.startsWith("https://plc.test/")) return new Response("{}", { status: 200 }); + return new Response("not found", { status: 404 }); +} + +function fakeAuth(): MiddlewareHandler { + return async (c, next) => { + const did = c.req.header("X-Test-Did"); + if (!did) return c.json({ error: "AuthRequired" }, 401); + c.set("serviceAuth", { issuer: did, audience: CONFIG.spaces!.authority!.serviceDid, lxm: undefined }); + await next(); + }; +} + +async function makeApp(): Promise<Hono> { + const db = createSqliteDatabase(":memory:"); + const resolved = resolveConfig(CONFIG); + const community = createCommunityIntegration({ db, config: resolved }); + await initSchema(db, resolved, { extraSchemas: [community.applySchema] }); + return createApp(db, resolved, { + spaces: { authMiddleware: fakeAuth() }, + community, + }); +} + +function call( + app: Hono, + method: string, + path: string, + did: string | null, + body?: any +): Promise<Response> { + const headers: Record<string, string> = {}; + if (did) headers["X-Test-Did"] = did; + if (body !== undefined) headers["Content-Type"] = "application/json"; + return app.fetch( + new Request(`http://localhost${path}`, { + method, + headers, + body: body !== undefined ? JSON.stringify(body) : undefined, + }) + ); +} + +describe("realtime + community", () => { + let app: Hono; + beforeAll(async () => { + app = await makeApp(); + }); + + it("community:<did> alias expands to reachable spaces", async () => { + // Adopt a community, create a child space, grant Charlie member. + const adoptRes = await call(app, "POST", "/xrpc/test.rt.community.adopt", ALICE, { + identifier: "did:plc:community", + appPassword: "anything", // mockFetch returns 200 for createSession + }); + expect(adoptRes.status).toBe(200); + const { communityDid } = (await adoptRes.json()) as any; + + const c1 = await call(app, "POST", "/xrpc/test.rt.community.space.create", ALICE, { + communityDid, + key: "general", + }); + expect(c1.status).toBe(200); + const general = ((await c1.json()) as any).space.uri as string; + + // Grant Charlie as member in general. + const g = await call(app, "POST", "/xrpc/test.rt.community.space.grant", ALICE, { + spaceUri: general, + subject: { did: CHARLIE }, + accessLevel: "member", + }); + expect(g.status).toBe(200); + + // Charlie mints a community-alias ticket; should expand to [space:general]. + const ticketRes = await call(app, "POST", "/xrpc/test.rt.realtime.ticket", CHARLIE, { + topic: `community:${communityDid}`, + }); + expect(ticketRes.status).toBe(200); + const body = (await ticketRes.json()) as any; + expect(body.topics).toContain(`space:${general}`); + }); +}); diff --git a/packages/contrail/package.json b/packages/contrail/package.json index 846e6e8..a64484d 100644 --- a/packages/contrail/package.json +++ b/packages/contrail/package.json @@ -78,7 +78,6 @@ "jiti": "^2.4.0" }, "devDependencies": { - "@atmo-dev/contrail-community": "workspace:*", "@cloudflare/workers-types": "^4.20250124.0", "@types/node": "^25.5.0", "@types/pg": "^8.20.0", diff --git a/packages/contrail/tests/realtime-e2e.test.ts b/packages/contrail/tests/realtime-e2e.test.ts index 0ef8aa2..01d1767 100644 --- a/packages/contrail/tests/realtime-e2e.test.ts +++ b/packages/contrail/tests/realtime-e2e.test.ts @@ -7,13 +7,10 @@ import { createApp } from "../src/core/router"; import { resolveConfig } from "../src/core/types"; import type { ContrailConfig } from "../src/core/types"; import type { RealtimeEvent } from "../src/core/realtime/types"; -import { createCommunityIntegration } from "@atmo-dev/contrail-community"; const ALICE = "did:plc:alice"; const BOB = "did:plc:bob"; -const CHARLIE = "did:plc:charlie"; -const MASTER_KEY = new Uint8Array(32).fill(5); const REALTIME_SECRET = new Uint8Array(32).fill(9); const CONFIG: ContrailConfig = { @@ -26,41 +23,12 @@ const CONFIG: ContrailConfig = { }, recordHost: {}, }, - community: { - masterKey: MASTER_KEY, - plcDirectory: "https://plc.test", - fetch: mockFetch, - resolver: mockResolver(), - }, realtime: { ticketSecret: REALTIME_SECRET, keepaliveMs: 60_000, }, }; -function mockResolver(): any { - return { - resolve: async (_did: string) => ({ - id: _did, - service: [ - { id: "#atproto_pds", type: "AtprotoPersonalDataServer", serviceEndpoint: "https://pds.test" }, - ], - }), - }; -} - -async function mockFetch(input: RequestInfo | URL, init?: RequestInit): Promise<Response> { - const url = typeof input === "string" ? input : input instanceof URL ? input.href : input.url; - if (url.endsWith("/xrpc/com.atproto.server.createSession")) { - return new Response( - JSON.stringify({ accessJwt: "a.b.c", refreshJwt: "r.r.r", did: "did:plc:community" }), - { status: 200, headers: { "content-type": "application/json" } } - ); - } - if (url.startsWith("https://plc.test/")) return new Response("{}", { status: 200 }); - return new Response("not found", { status: 404 }); -} - function fakeAuth(): MiddlewareHandler { return async (c, next) => { const did = c.req.header("X-Test-Did"); @@ -73,11 +41,9 @@ function fakeAuth(): MiddlewareHandler { async function makeApp(): Promise<Hono> { const db = createSqliteDatabase(":memory:"); const resolved = resolveConfig(CONFIG); - const community = createCommunityIntegration({ db, config: resolved }); - await initSchema(db, resolved, { extraSchemas: [community.applySchema] }); + await initSchema(db, resolved); return createApp(db, resolved, { spaces: { authMiddleware: fakeAuth() }, - community, }); } @@ -280,37 +246,4 @@ describe("realtime e2e (in-memory pubsub, SSE transport)", () => { ); expect(res.status).toBe(401); }); - - it("community:<did> alias expands to reachable spaces", async () => { - // Adopt a community, create two child spaces, grant Charlie member on one. - const adoptRes = await call(app, "POST", "/xrpc/test.rt.community.adopt", ALICE, { - identifier: "did:plc:community", - appPassword: "anything", // mockFetch returns 200 for createSession - }); - expect(adoptRes.status).toBe(200); - const { communityDid } = (await adoptRes.json()) as any; - - const c1 = await call(app, "POST", "/xrpc/test.rt.community.space.create", ALICE, { - communityDid, - key: "general", - }); - expect(c1.status).toBe(200); - const general = ((await c1.json()) as any).space.uri as string; - - // Grant Charlie as member in general. - const g = await call(app, "POST", "/xrpc/test.rt.community.space.grant", ALICE, { - spaceUri: general, - subject: { did: CHARLIE }, - accessLevel: "member", - }); - expect(g.status).toBe(200); - - // Charlie mints a community-alias ticket; should expand to [space:general]. - const ticketRes = await call(app, "POST", "/xrpc/test.rt.realtime.ticket", CHARLIE, { - topic: `community:${communityDid}`, - }); - expect(ticketRes.status).toBe(200); - const body = (await ticketRes.json()) as any; - expect(body.topics).toContain(`space:${general}`); - }); }); diff --git a/packages/contrail/vitest.config.ts b/packages/contrail/vitest.config.ts index 70a22e6..04f1a15 100644 --- a/packages/contrail/vitest.config.ts +++ b/packages/contrail/vitest.config.ts @@ -1,5 +1,4 @@ import { defineConfig } from "vitest/config"; -import path from "node:path"; export default defineConfig({ test: { @@ -7,15 +6,4 @@ export default defineConfig({ // PostgreSQL tests share a single database and cannot run in parallel fileParallelism: false, }, - resolve: { - alias: { - // Point at contrail-community's source so tests don't require a built - // dist. Mirrors the contrail-community package's own vitest alias for - // `@atmo-dev/contrail` → contrail/src. - "@atmo-dev/contrail-community": path.resolve( - __dirname, - "../contrail-community/src/index.ts" - ), - }, - }, }); diff --git a/packages/lexicons/tests/generate.test.ts b/packages/lexicons/tests/generate.test.ts index 44f45c5..eaf57bc 100644 --- a/packages/lexicons/tests/generate.test.ts +++ b/packages/lexicons/tests/generate.test.ts @@ -273,7 +273,10 @@ describe("extractXrpcMethods / listXrpcMethods", () => { const config: ContrailConfig = { namespace: "test.comm", collections: { message: { collection: "app.event.message" } }, - spaces: { type: "tools.atmo.event.space", serviceDid: "did:web:test.example#svc" }, + spaces: { + authority: { type: "tools.atmo.event.space", serviceDid: "did:web:test.example#svc" }, + recordHost: {}, + }, community: { masterKey: new Uint8Array(32).fill(1) }, realtime: { ticketSecret: new Uint8Array(32).fill(2) }, }; diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 11a1eb6..48ffc2e 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -386,9 +386,6 @@ importers: specifier: ^2.4.0 version: 2.6.1 devDependencies: - '@atmo-dev/contrail-community': - specifier: workspace:* - version: link:../contrail-community '@cloudflare/workers-types': specifier: ^4.20250124.0 version: 4.20260424.1 diff --git a/refs/spaces-later.md b/refs/spaces-later.md index 270b55c..47a0714 100644 --- a/refs/spaces-later.md +++ b/refs/spaces-later.md @@ -4,26 +4,65 @@ Deferred items from the spaces design review. Not blocking shipping; keep an eye on these as usage grows or as the permissioned-data spec firms up. See also [spaces-spec-mapping.md](./spaces-spec-mapping.md). +Items resolved by the six-phase refactor (credential flow, host/authority +split, enrollment, community as separate package, etc.) have been removed +from this list — see the spec-mapping doc for the post-refactor state. + ## Hydrated members endpoint -`space.listMembers` today returns raw `{did, perms, addedAt, addedBy}` rows. + +`space.listMembers` today returns raw `{did, addedAt, addedBy}` rows. Every client ends up wanting profile hydration (handle, displayName, avatar). Add `space.getMembers` (or extend `listMembers`) with: - Cursor-based pagination (current endpoint is unbounded) - Optional `hydrate=true` that joins against the configured profile collection - Sort options (joined-at, alphabetical by handle) +## DID-doc publication helper + +The authority's signing key needs to be published in its DID document under +`#atproto_space_authority` (verification method) — that's how external +verifiers find it. Today the deployer does this manually: +- For `did:web:contrail.example.com`: edit `.well-known/did.json`. +- For `did:plc`: a PLC operation signed with the rotation key. + +A `contrail authority publish-key` CLI subcommand could: +- For `did:web`, emit the verification-method JSON to stdout for the deployer to drop into their DID doc. +- For `did:plc`, build and submit the PLC operation. + +Without this, external verifiers can't validate the authority's credentials. +The in-process default works either way (host has the key directly). + +## Auto-wiring discovery resolvers + +`createPdsBindingResolver` and `createDidDocBindingResolver` are exported but +not wired into the default verifier. Today the in-process verifier uses only +`Local` (configured authority) + `Enrollment` (locally consented spaces). + +For deployments that want to accept credentials from any authority that's +properly bound on a user's PDS or DID doc, the deployer composes the +resolvers manually (see [deployment-shapes](../docs/10-deployment-shapes.md)). + +A higher-level config knob — `spaces.recordHost.acceptExternalAuthorities: true` +or similar — could auto-wire the full discovery chain. Decide whether the +fast-path (Local+Enrollment only) or the universal-path (full chain) is the +right default once we have real cross-host deployments. + ## More tests -The e2e + invite tests cover the happy paths. Gaps: -- Non-owner calling `createSpace` (should succeed — anyone can create their - own) vs non-member trying to use someone else's space URI -- App policy enforcement in both `allow` and `deny` modes (clientId checks) -- `deleteRecord` by owner on another author's record -- Re-querying a soft-deleted space returns NotFound -- `leaveSpace` by owner (should error) -- `whoami` for owner, member, non-member + +Phase 3-5 added good coverage for credentials, binding, enrollment. Gaps that +predate the refactor and still apply: + +- Non-owner calling `createSpace` (should succeed — anyone can create their own). +- App policy enforcement in both `allow` and `deny` modes (clientId checks). +- `deleteRecord` by owner on another author's record (should fail). +- Re-querying a soft-deleted space returns NotFound. +- `leaveSpace` by owner (should error). +- `whoami` for owner, member, non-member, with and without community integration. ## Config-change behavior + What happens today if a deployment: + - Adds a new collection after spaces already contain data? The per-collection table (`spaces_records_<short>`) won't exist until schema init re-runs. `listCollections` swallows the missing-table error, but `putRecord` / @@ -36,42 +75,80 @@ What happens today if a deployment: Need a config-drift audit (or migration) story. ## Verify `clientId` actually flows through + `checkAccess` uses `ServiceAuth.clientId` for app policy checks. Confirm: + - JWT verifier actually extracts `client_id` from real atproto service tokens - (not just our test fixture) -- App policy with a populated `apps[]` blocks/allows correctly in practice -- Empty `apps[]` under `mode: "deny"` blocks everyone (is that what we want?) + (not just our test fixture). +- App policy with a populated `apps[]` blocks/allows correctly in practice. +- Empty `apps[]` under `mode: "deny"` blocks everyone (is that what we want?). If `clientId` is `undefined` in the wild, app policy is decorative. +App policy is also currently checked at credential-issuance time but not +enforced again on the record host. For very long-lived credentials (>2h), an +app removed from the allowlist could continue acting until expiry. The TTL is +the spec's revocation bound; live with it. + ## Join requests (spec-adjacent, not in spec) + The rough spec punts invite/onboarding mechanics to apps. A natural fit given our invite system: a fourth kind `request` where `redeem` creates a pending row for the owner to approve. Likely wants: + - `space.requestJoin` → creates pending row - `space.listJoinRequests` (owner) → pending rows - `space.approveJoinRequest` / `space.denyJoinRequest` -Should this live under `space.*` or a separate extras namespace? +Should this live under `<ns>.space.*` or `<ns>.spaceExt.*`? + +## Authority migration + +A space's authority can change in principle — the `recordHost.enroll` row maps +`spaceUri → authorityDid`, re-enroll with a new authority and credentials +from the new authority will start verifying. But: + +- Existing credentials from the old authority don't auto-revoke; they expire + within their TTL. +- The PDS-record / DID-doc discovery sources (if used) need updating in lockstep. +- No helper API for this — the deployer or owner does each step manually. -## Real-time: SSE / subscriptions -Every collaborative app wants "new records in this space, as they land." -The spec's sync model uses write-notifications through the space owner; we -don't have that yet. Lightweight interim: Server-Sent Events on -`space.subscribeRecords?spaceUri=&collection=`. Works for first-party apps -right away; swap to the real thing later. +A `<ns>.recordHost.transferAuthority` endpoint could automate the wire-level +parts (re-enroll, optionally short-circuit credential cache). -## Namespace split for contrail-specific extras — done -`space.invite.*` and `space.whoami` moved to `<ns>.spaceExt.*`. See -[spaces-spec-mapping.md § Contrail extras](./spaces-spec-mapping.md#contrail-extras-namespace-nsspaceext). -`leaveSpace` is still in `space.*` — revisit if the spec lands with different -self-remove semantics. +## Multi-authority spaces + +In principle the architecture allows several authorities to all sign for one +space (replication / failover scenarios). The host's enrollment is 1:1 today +(one authority per space) but could become 1:N with a small schema change. +Spec is silent. Defer until a real use case. ## Ownership transfer -Dropped for now. The space URI is `at://<ownerDid>/<type>/<key>` — owner DID + +Dropped for now. The space URI is `ats://<ownerDid>/<type>/<key>` — owner DID is baked into the URI, and every record/member/invite row keys off that URI string. Transferring would mean either rewriting every referencing row in a transaction (and breaking external refs to the old URI) or decoupling storage from URI with an internal stable space id (bigger refactor). Revisit once the spec pins down whether ownership transfer exists and what the URI authority is supposed to be post-transfer. + +## Real-time over credentials + +Realtime tickets are signed by `realtime.ticketSecret`, not by the space +authority. If the host/authority split goes far enough that they're operated +by different parties, the realtime ticket model may need rethinking — does +the host mint tickets it then validates itself, or does the authority issue +realtime grants the host honors? Today both run in one process so it doesn't +matter. + +--- + +## Resolved by the six-phase refactor (kept for history) + +- ~~Space-credential flow~~ — done in phase 3. +- ~~Binding resolution (PDS records, DID-doc service entries)~~ — done in phase 4. +- ~~Independent host/authority deployments~~ — done in phase 5. +- ~~Real-time SSE / subscriptions~~ — landed via the realtime module. +- ~~Namespace split for contrail-specific extras~~ — `<ns>.spaceExt.*` shipped. +- ~~Community as separate package~~ — done in phase 6 (`@atmo-dev/contrail-community`). diff --git a/refs/spaces-spec-mapping.md b/refs/spaces-spec-mapping.md index 293a425..70d697b 100644 --- a/refs/spaces-spec-mapping.md +++ b/refs/spaces-spec-mapping.md @@ -7,70 +7,71 @@ is this doc. The goal is to make it obvious, when the real spec lands, where contrail already lines up and where it needs to change. Contrail is a backend-in-a-bottle / simple appview, not a PDS. For permissioned -data it currently stores everything in its own database; the plan is to -switch permissioned reads to come from users' PDSes once the protocol-level -flow is shipped (same story we already have for public records via jetstream). +data it currently stores everything in its own database; the long-term plan +is to switch permissioned reads to come from users' PDSes once the +protocol-level flow is shipped (same story we already have for public records +via jetstream). + +The spaces implementation went through a six-phase refactor (phases 1–6, +documented in conversation history) that aligned the architecture with the +spec and split community out into its own package. This doc reflects the +post-refactor state. --- ## Concept-by-concept alignment -| Spec concept | Contrail | Alignment | Notes | -| ------------------------------ | ------------------------------------------------------------ | --------- | ------------------------------------------------------------------------------ | -| Space owner (DID) | `spaces.owner_did` | ✅ | 1:1 | -| Space type (NSID) | `spaces.type` | ✅ | 1:1 | -| Space key / skey | `spaces.key` | ✅ | TID-generated when caller omits it | -| Record addressing 6-tuple | `(owner, type, key, author-did, collection, rkey)` | ✅ | Storage is keyed by `(space_uri, did, rkey)`; `space_uri` encodes the first 3 | -| Single ACL = member list | `spaces_members (did)` | ✅ | Membership is binary: you're in or you're out. Owner is implicit member. No read/write tiering — apps filter writes themselves | -| Space credential (2–4h token) | _none; service-auth JWTs used directly_ | ❌ | Fine while contrail is a single appview. Add a shim when real PDS sync lands | -| App allow/deny | `appPolicy {mode, apps[]}` | ✅ | Matches spec's default-allow / default-deny model. Visible only to the owner | -| Permissioned repo per user | single DB (`spaces_records_<short>`) | ⚠️ | Structurally compatible — keyed per `(space, author)`. Federation is future | -| ECMH commit / sync log | _none_ | ❌ | Out of scope until federated sync exists | -| Pull-based sync, write notifs | _none_ | ❌ | Same | -| URI scheme | `at://<owner>/<type>/<key>` for spaces; records not exposed | ⚠️ | Spec floats `ats://`. We centralize construction in `src/core/spaces/uri.ts` | -| Authority model for record URI | sidestepped (records keyed, not URI-addressed) | ✅ | Spec is undecided; we don't commit either way | -| Managing app routing | _none (join-requests etc. not modeled yet)_ | ⚠️ | See [spaces-later.md](./spaces-later.md) | +| Spec concept | Contrail | Alignment | Notes | +| ------------------------------ | --------------------------------------------------------------- | --------- | ----------------------------------------------------------------------------------------------------------- | +| Space owner (DID) | `spaces.owner_did` | ✅ | 1:1 | +| Space type (NSID) | `spaces.type` | ✅ | 1:1 | +| Space key / skey | `spaces.key` | ✅ | TID-generated when caller omits it | +| Record addressing 6-tuple | `(owner, type, key, author-did, collection, rkey)` | ✅ | Storage is keyed by `(space_uri, did, rkey)`; `space_uri` encodes the first 3 | +| `ats://` URI scheme | `ats://<owner>/<type>/<key>` | ✅ | Centralized in `src/core/spaces/uri.ts` | +| Single ACL = member list | `spaces_members (did)` | ✅ | Membership is binary. Owner is implicit member. No read/write tiering. | +| Member list `(did, read\|write)` tuples | binary membership only | ⚠️ | Spec says read/write tiered. Pragmatic divergence; flip if/when spec firms | +| Member list as PDS record | server-side state on authority | ⚠️ | Spec says published on owner's PDS, synced. Future migration; the authority caches today | +| Space credential (2–4h token) | ES256 JWTs via `<ns>.space.getCredential` / `refreshCredential` | ✅ | Phase 3. Default 2h TTL. `iss` = authority DID; signed with the authority's published key | +| Credential signed by owner key | signed by **issuer** DID; binding from PDS record / DID-doc / owner-self | ⚠️ ext | Spec says owner-key. We extend so user-owned DIDs can authorize a separate issuer without DID-doc surgery | +| App allow/deny | `appPolicy {mode, apps[]}` | ✅ | Matches default-allow / default-deny. Checked at credential issuance only | +| Discovery via DID doc | `#atproto_space_authority` service entry resolver | ✅ | Phase 4. Plus PDS-record fallback (extension) and owner-self fallback | +| Permissioned repo per user | single DB (`spaces_records_<short>`) | ⚠️ | Structurally compatible — keyed per `(space, author)`. Federation is future | +| ECMH commit / sync log | _none_ | ❌ | Out of scope until federated sync exists | +| Pull-based sync, write notifs | _none_ | ❌ | Same | +| Authority model for record URI | sidestepped (records keyed, not URI-addressed) | ✅ | Spec is undecided; we don't commit either way | +| Managing app routing | _none (join-requests etc. not modeled yet)_ | ⚠️ | See [spaces-later.md](./spaces-later.md) | +| Host/AppView split | `spaces.authority` + `spaces.recordHost` independently runnable | ➕ | Phase 5. Spec implies but doesn't fully model. See [deployment-shapes](../docs/10-deployment-shapes.md) | +| Enrollment as host consent | `<ns>.recordHost.enroll` + `record_host_enrollments` table | ➕ | Phase 5. Spec doesn't address consent — we add explicit binding registration | + +Legend: ✅ aligned · ⚠️ pragmatic divergence · ❌ unimplemented (deliberate) · ➕ extension over the spec --- ## Endpoints -All endpoints are emitted under `<config.namespace>.space.*` from templates in -`lexicon-templates/spaces/`. - -### Read -- `space.listSpaces` — caller's spaces (scope=member|owner) -- `space.getSpace` — metadata; supports `?inviteToken=` bearer read -- `space.listMembers` — members for a space (member/owner only) -- `space.listRecords` — space-scoped record listing; bearer-read supported -- `space.getRecord` — single record; bearer-read supported - -### Write -- `space.putRecord` -- `space.deleteRecord` - -### Owner-gated (space management) -- `space.createSpace` -- `space.addMember` -- `space.removeMember` -- `space.leaveSpace` — self-remove; owner cannot leave (extra) - -### Contrail extras (namespace: `<ns>.spaceExt.*`) -Clearly-off-spec features live under a separate namespace so the `space.*` -surface stays close to whatever the permissioned-data spec becomes. Moved here -from `space.*` in an earlier refactor. - -- `spaceExt.whoami` — caller's relationship to a space (owner / member flags) -- `spaceExt.invite.create` — returns raw token once; hash stored -- `spaceExt.invite.redeem` -- `spaceExt.invite.list` -- `spaceExt.invite.revoke` - -Invites have three kinds: `join`, `read`, `read-join`. `read` tokens grant -bearer-only anonymous read access; `read-join` does both; `join` requires a -signed-in caller and grants a membership row. None of this is in the spec — -it lives here because the spec explicitly defers invite/onboarding mechanics -to apps, and shipping a working invite primitive is useful for every consumer. +All endpoints are emitted under `<config.namespace>.*` from templates in +`packages/lexicons/lexicon-templates/`. + +### Authority (`<ns>.space.*` — spec-aligned) +- `createSpace` `getSpace` `listSpaces` `deleteSpace` +- `listMembers` `addMember` `removeMember` `leaveSpace` +- `getCredential` `refreshCredential` + +### Record host +- `<ns>.space.putRecord` `deleteRecord` `getRecord` `listRecords` +- `<ns>.space.uploadBlob` `getBlob` `listBlobs` (optional) +- `<ns>.recordHost.enroll` + +### Contrail extras (`<ns>.spaceExt.*`) +Clearly-off-spec features that don't map cleanly to the rough spec. +- `whoami` — caller's relationship to a space (owner / member / extension fields) + +### Invites (`<ns>.invite.*`) +- `create` `redeem` `revoke` `list` + +Invites have three kinds: `join`, `read`, `read-join`. Spec defers +invite/onboarding mechanics to apps; we ship a working primitive because +every consumer needs one. ### Collection integration Per-collection `listRecords` / `getRecord` accept `?spaceUri=` (space-scoped) @@ -79,44 +80,54 @@ public + own-member-spaces union (see `src/core/router/collection.ts`). --- +## What changed in the six-phase refactor + +| Phase | Brought us | Notes | +|---|---|---| +| 1 | `SpaceAuthority` + `RecordHost` interface boundary | Pure refactor; `StorageAdapter` is their union | +| 2 | Spaces no longer imports community | Whoami extension hook + `CommunityInviteHandler` interface | +| 3 | Credential issuance + verification | ES256 JWTs, `X-Space-Credential` header, in-process verifier | +| 4 | Binding resolution | PDS-record + DID-doc resolvers; `iss != owner` allowed via the binding | +| 5 | Independent deployment + enrollment | Authority and host runnable as separate processes; `recordHost.enroll` consent | +| 6 | Community as separate package | `@atmo-dev/contrail-community` with integration interface | + +After phase 6 the architecture maps to the spec roughly as: + +``` + spec concept contrail mapping + ──────────── ──────────────── + "space host" ─→ space authority (signs creds, holds members) + "permissioned repo" ─→ record host (stores records, enrolls spaces) + "external space hosts" ─→ binding resolver chain (multi-authority support) + "managing app routing" ─→ not yet (deferred — spaces-later.md) +``` + +--- + ## Migration readiness -Hasn't shipped yet → nothing to migrate, but the shape of what changes when -the real spec lands: - -1. **URI scheme swap (if any).** Centralized in `src/core/spaces/uri.ts` — - flip `at://` to `ats://` (or whatever) in two helpers and every caller - follows. -2. **Space-credential flow.** Needs an endpoint that mints short-lived tokens - from an owner key, and a verifier that accepts them in place of a - service-auth JWT on read paths. The current JWT middleware - (`src/core/spaces/auth.ts`) is the right anchor for this. -3. **Read records from PDSes.** Mirrors the jetstream ingestion we already do - for public data: consume permissioned-repo sync, write into the same - `spaces_records_<short>` tables. The storage schema is already keyed per - `(space, author)` so no migration needed on that side. -4. **ECMH commits & sync log.** Greenfield; unrelated to existing storage. -5. **Endpoint naming.** Spec doesn't pin XRPC names. When it does, rename - lexicon template files + routes. No storage churn. - -### Design decisions worth preserving -- Keep the member list as the single ACL. Don't add roles or per-collection - policies just because it's easy — the spec is emphatic that the member list - is _the_ ACL. -- Membership is binary, not tiered. Previously had `perms: "read" | "write"` - per member row; collapsed to plain membership because the rough spec is - moving toward "member = access, apps filter writes." Delete keeps the - owner / own-record rule, but that's about *which records you can affect*, - not a permission tier on the member row. -- Don't over-engineer the space row with pre-emptive extension columns. - Previously had `member_list_ref` as a hook for externally-managed - membership; dropped because the community-module case is handled via - ownership (community-owned spaces are managed by the community module, no - flag column needed). If a future need for external membership sources - shows up, add the column then. -- Keep `space.whoami`, `space.leaveSpace`, and the invite endpoints clearly - labeled as contrail extras in docs. If the spec ends up naming some of - them, renaming is cheap; relying on them from the base spec isn't. -- Don't mint a canonical record URI. The spec is undecided on the authority - (user DID vs space owner DID); storing records by tuple avoids picking. +What still needs to change when the real spec lands: + +1. **Member list moves to PDS records.** The spec says it's a record on the owner's PDS, synced. Our authority holds it server-side. Future: a watcher consumes member-list records via Jetstream and reconciles into `spaces_members`. Auth-side `addMember` becomes a PDS write rather than an internal API. + +2. **Records federate from user PDSes.** Today the record host *is* the source of truth. When permissioned-repos ship, records federate; the host becomes an aggregator. Storage schema (`spaces_records_<short>`, keyed per `(space, author)`) already supports this — the change is in the write path, not the read path. + +3. **ECMH commits & sync log.** Greenfield. Required for federation. + +4. **Endpoint naming.** Spec doesn't pin XRPC names. When it does, rename lexicon template files + routes. No storage churn. + +5. **Possibly: `(did, read|write)` member tuples.** If the spec stays at tiered membership and doesn't move to binary, add an `access` column on `spaces_members` and branch the ACL check in `acl.ts`. One-day change. + +6. **Possibly: credential `iss = owner DID`.** If the spec forbids the issuer-DID indirection we use for user-owned DIDs, fall back to "owner adds a host-controlled verification method to their DID doc" (HappyView's hidden assumption). Operationally heavier; we hold the looser reading until forced to tighten. + +--- + +## Design decisions worth preserving +- **Keep the member list as the single ACL.** Don't add roles or per-collection policies just because it's easy — the spec is emphatic that the member list is _the_ ACL. +- **Membership is binary, not tiered.** Previously had `perms: "read" | "write"` per member row; collapsed to plain membership because the rough spec is moving toward "member = access, apps filter writes." Delete keeps the owner / own-record rule, but that's about *which records you can affect*, not a permission tier on the member row. +- **Don't over-engineer the space row with pre-emptive extension columns.** Previously had `member_list_ref` as a hook for externally-managed membership; dropped because the community-module case is handled via ownership (community-owned spaces are managed by the community module, no flag column needed). If a future need for external membership sources shows up, add the column then. +- **Keep `spaceExt.whoami`, `space.leaveSpace`, and the invite endpoints clearly labeled as contrail extras in docs.** If the spec ends up naming some of them, renaming is cheap; relying on them from the base spec isn't. +- **Don't mint a canonical record URI.** The spec is undecided on the authority (user DID vs space owner DID); storing records by tuple avoids picking. +- **Enrollment is the host's source of truth.** Even when PDS records and DID-doc service entries declare authority bindings, the host's local enrollment is what actually gates record acceptance. Keeps the host's consent explicit and prevents abuse of the open-ended discovery layer. +- **Keep the issuer-DID indirection as an extension, not a hard architectural choice.** The credential verifier supports `iss == owner` (literal-spec) and `iss != owner` (with binding). If the spec forbids the latter, we degrade gracefully.