diff --git a/README.md b/README.md index c3bad76..4b0ad43 100644 --- a/README.md +++ b/README.md @@ -6,6 +6,9 @@ an [AT Protocol](https://atproto.com) personal data server in Zig, built on **live:** [pds.zat.dev](https://pds.zat.dev) +[Spaces support](docs/permissioned-data.md) is experimental; protocol updates can +require coordinated client migrations. + > featured in the [atproto spaces alpha](https://atproto.com/blog/atproto-spaces-alpha). > name credit: jim (`calabro.io`) suggested `zds`. diff --git a/docs/architecture.md b/docs/architecture.md index 223e4df..bafc4ad 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -29,8 +29,11 @@ Routing, `/api`, and `/api/openapi.json` use the same comptime endpoint inventor - Dependency fixes should happen at the owning layer. Since this project also maintains `zat`, ZDS should consume a fixed `zat` commit rather than vendoring or patching a private copy. +- Spaces HTTP-signature parsing and profile validation stay in + `src/internal/space_signature.zig`; `zat` supplies DID-key decoding and JOSE + verification. This is a narrow Spaces profile, not a general HTTP-signature API. - SQLite stores account, repo, commit, token, OAuth, blob metadata, and identity - state. + state, plus Spaces revocations, authority sequencing, and pending notifications. - Password sessions are durable rows keyed by access and refresh JWT IDs. Refresh rotates the stored token family instead of trusting self-contained JWTs alone. - Account-plane audit events store subject, actor, and optional controller DID @@ -71,14 +74,16 @@ requests. `ZDS_CRAWLERS` should include both `https://bsky.network` and OAuth follows the ATProto OAuth profile: PAR is required, redirect URIs are validated against client metadata, permission-set includes must resolve, tokens -enforce granular repo/blob/rpc/account/identity scopes, DPoP-bound OAuth access +enforce granular repo/blob/rpc/account/identity/space scopes, DPoP-bound OAuth access tokens must present matching proof headers on resource requests, and revocation affects resource-server checks. Redirect matching is exact, with the RFC 8252 loopback exception the reference provider applies: a client whose metadata says `application_type: native` and registers `http://127.0.0.1/callback` (or the `[::1]` form) without a port matches that redirect on whatever port it bound at request time. A registered port still has to match and `localhost` is not a -loopback host. +loopback host for this redirect exception. Separately, the virtual OAuth client +ID `http://localhost` supports the ATProto localhost development profile, with +loopback-IP redirect URIs and metadata supplied in its query string. Password sessions and app passwords match the reference PDS account model. Session JWTs are accepted only when their JTI is present in the active session @@ -96,7 +101,7 @@ login is supported so an account can be chosen by the authenticator during the OAuth flow. `/account` is the resident management surface. Its security section, -`/account/security`, manages passkeys and app passwords with account-password +`/account/manage`, manages passkeys and app passwords with account-password authentication, matching the official PDS boundary that app passwords cannot mint or revoke other app passwords. diff --git a/docs/deployment.md b/docs/deployment.md index 7f36f57..89a430e 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -50,7 +50,8 @@ Required checks: ```sh just test -just smoke +just smoke-all +zig fmt --check build.zig build.zig.zon src bench tools git diff --check zig zen ``` @@ -62,17 +63,41 @@ storage, repo, blob, proxy, or permissioned-data work. ## manual deployment -The manual path is for an operator already authenticated to the Fly app: +The manual path is for an operator already authenticated to the Fly app. +Push the release first. If no pipeline starts, verify that absence on the +spindle, run the same checks locally (including the ARMv6 command below), and +deploy a clean archive of the pushed revision so local `.env` and workspace +files are not uploaded to the remote builder: ```sh flyctl auth whoami -flyctl deploy --remote-only --app zds-pds +release_src=$(mktemp -d) +git archive HEAD | tar -x -C "$release_src" +(cd "$release_src" && flyctl deploy --remote-only --app zds-pds) ``` Do not create a second configuration path in shell history. `fly.toml` remains the source of non-secret deployment configuration, and Fly secrets remain the source of secret configuration. +### October 1 deployment observation + +The v0.4.0 push (`8d0deeb2cf5c`) reached the knot, but no matching spindle +pipeline appeared. `tg pipeline status zat.dev/zds` failed resolving the owner +repo record; the spindle still reported `f68da7c` as its latest run. The release +used the manual path after all local CI-equivalent checks passed. Restoring the +push trigger remains an operations follow-up; a successful Git push alone must +not be treated as a successful deployment. + +For this repo, query the spindle directly if the CLI lookup fails: + +```sh +curl -fsSG https://spindle.tangled.sh/xrpc/sh.tangled.ci.queryPipelines \ + --data-urlencode 'repo=did:plc:6atzzmsifkfmfqmeqrl6jwzo' \ + --data-urlencode "commits=$(git rev-parse HEAD)" \ + --data-urlencode 'limit=5' +``` + ## verification Capture the current health response before deploying so there is a baseline: diff --git a/docs/operations.md b/docs/operations.md index b96f584..df874f3 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -207,9 +207,9 @@ Publish commands: ```sh just docker-publish -just docker-publish v0.3.0 +just docker-publish vX.Y.Z just docker-publish-current -just docker-publish-release v0.3.0 +just docker-publish-release vX.Y.Z ``` ## releases @@ -226,7 +226,8 @@ version check. Release checklist: ```sh just test -just smoke +just smoke-all +zig fmt --check build.zig build.zig.zon src bench tools git diff --check zig zen git tag -a vX.Y.Z -m "vX.Y.Z" diff --git a/docs/permissioned-data-proposal-94.md b/docs/permissioned-data-proposal-94.md index 088d666..6de66a1 100644 --- a/docs/permissioned-data-proposal-94.md +++ b/docs/permissioned-data-proposal-94.md @@ -1,5 +1,9 @@ # permissioned data proposal 94 notes +Historical design notes: credential and membership examples below predate +the October 1, 2026 alpha. Use [permissioned data](permissioned-data.md) for the +current ZDS contract. + These notes track Bluesky proposal PR #94 from the perspective of ZDS. They are not implementation instructions and should not be treated as a stable protocol contract. The proposal is a draft; use this page to reduce wrong-way risk while @@ -9,8 +13,9 @@ PR: ## working posture -ZDS is a sandcastle for this surface. It is acceptable to make subtractive -changes while there are no external production users, but avoid deepening older +The original exploration treated this surface as a sandcastle without external +production users. That assumption no longer describes the deployment: consumers +need coordinated migrations for breaking changes. Avoid deepening older experimental shapes when the proposal has already moved away from them. Prefer small alignment work that preserves optionality: diff --git a/docs/permissioned-data.md b/docs/permissioned-data.md index 64a20ca..c8e6579 100644 --- a/docs/permissioned-data.md +++ b/docs/permissioned-data.md @@ -1,8 +1,8 @@ # permissioned data Permissioned data support is experimental and operator gated. Treat the current -ZDS surface as a prototype for local experiments, not as a claim of parity with -the evolving upstream permissioned-data proposal. ZDS documents the +ZDS surface as an alpha implementation that can change incompatibly, not as a +claim of complete parity with the evolving upstream proposal. ZDS documents the `com.atproto.space.*` routes as experimental, and only enables the handlers when explicitly configured: @@ -85,7 +85,7 @@ ZDS splits protocol data routes from baseline PDS management routes: - `com.atproto.space.*`: `getSpace`, `listSpaces`, `listRepos`, `getDelegationToken`, `getSpaceCredential`, records, blobs, signed commits, full and incremental repo sync, blob enumeration, notification registration - and removal, write notifications, and deletion notifications + and removal, write notifications, credential revocation, and deletion notifications - `com.atproto.simplespace.*`: `createSpace`, `getSpace`, `updateSpace`, `deleteSpace`, `putMember`, `removeMember`, `listMembers`, and the managing-app `checkUserAccess` hook @@ -219,7 +219,7 @@ space host, and account PDS. ## client-visible contract (verified across implementations, 2026-08-21) -plyr.fm (#1897, #1898) now depends on the behaviours below. They were checked +This historical comparison informed plyr.fm (#1897, #1898). The behaviours were checked against the reference PDS (`permissioned-data` branch), rsky, and atproto-crates so that nobody mistakes them for ZDS quirks and "cleans them up". @@ -235,6 +235,10 @@ Clients must not infer capability from `scopes_supported`: the grant on the issued token is the only portable signal. `listMembers` is the one row where implementations differ; a client should treat it as owner-only. +The September policy update supersedes the ZDS `listMembers` scope cell above: +ZDS now requires the owner and a covering read grant. Other implementations +in this historical table have not been rechecked for the October alpha. + ## storage shape ZDS stores many actor repos in one SQLite database, so permissioned data uses @@ -252,7 +256,7 @@ explicit space-scoped tables instead of the public repo tables: - `permissioned_space_repos`: repo-host state, including the full LtHash state needed to update and sign that writer's local permissioned repo - `permissioned_space_writers`: authority-host writer registry, containing only - each known writer's latest revision and 32-byte commit digest + each known writer's latest repo revision, space revision, and 32-byte commit digest - `permissioned_space_record_oplog`: incremental record changes by `(space, repo_did, rev, idx)` - `permissioned_space_notify_registrations`: expiring service-identified @@ -405,7 +409,7 @@ not learn permissioned-data special cases unless the protocol requires a shared primitive. Prefer a permissioned-data helper over scattering conditionals into stable PDS code. -## October 1 local upgrade +## October 1 upgrade Startup creates the revocation, sequence, and notification tables and backfills existing writer rows with ordered space revisions. Existing records, blobs, @@ -417,8 +421,8 @@ DPoP scheme and `type` request field are not compatibility paths. The local consumers requiring follow-up are plyr.fm's Python client, Doodl's browser/live/preview services, and Lore's extension and website. Doodl and Lore also need the prior release's policy split and all three still have membership -callers to update. This document records the local implementation target, not -a production deployment. +callers to update. These are the downstream migration requirements for v0.4.0; +ZDS does not translate the previous wire contract for old clients. The optional SDK interop lane runs against an externally installed October 1 `@atproto/space` snapshot: diff --git a/docs/references.md b/docs/references.md index 4e4c7ce..e382451 100644 --- a/docs/references.md +++ b/docs/references.md @@ -143,8 +143,9 @@ Links: [permissioned-data branch](https://github.com/bluesky-social/atproto/tree/permissioned-data), [Bulletin](https://github.com/bluesky-social/bulletin) -The lexicons under `lexicons/com/atproto/{space,simplespace}` on the branch are -the contract for `com.atproto.space.*` and `com.atproto.simplespace.*`; ZDS is +Use the alpha snapshot pinned in [permissioned data](permissioned-data.md), +including its `lexicons/com/atproto/{space,simplespace}` definitions, as the +contract for `com.atproto.space.*` and `com.atproto.simplespace.*`; ZDS is listed in the announcement as a compatible PDS. Bulletin is the smallest real client and the fastest interop check: sign in from a zds account and create a board. See [permissioned data](permissioned-data.md) for the ZDS-side notes. diff --git a/docs/simplespace-upgrade.md b/docs/simplespace-upgrade.md index f469b20..cae8a89 100644 --- a/docs/simplespace-upgrade.md +++ b/docs/simplespace-upgrade.md @@ -1,6 +1,6 @@ # Simplespace independent read/write access -ZDS follows the September 9, 2026 permissioned-data lexicons ([upstream change](https://github.com/bluesky-social/atproto/commit/1ce408be0edc71778a80eb0aa744812cfcb67390)). This experimental API update intentionally removes the previous request shapes. +The policy split follows the September 9, 2026 lexicons ([upstream change](https://github.com/bluesky-social/atproto/commit/1ce408be0edc71778a80eb0aa744812cfcb67390)). ZDS v0.4.0 also adopts the October 1 credential, notification, and `spaceType` changes; see the [current Spaces contract](permissioned-data.md). These experimental updates intentionally remove the previous request shapes. ## Client changes @@ -30,9 +30,9 @@ For example, a public-readable space with selected writers: ``` Read policy governs credential issuance. Write policy governs which writers the -space authority tracks and forwards notifications for. ZDS also checks write -policy synchronously when it hosts the authority and another local account -writes into the space. The authority retains access to its own space. +space authority tracks and forwards notifications for. A resident can store +records in their own repo even when the space denies write access; those writes do not register the writer or get forwarded by the +authority. This also applies when both roles are hosted by ZDS. ## Stored data @@ -46,7 +46,9 @@ Startup migrates the previous SQLite schema in one transaction: This preserves previous access while removing compatibility code from the API. Old clients must update; no legacy route or single-policy translation remains. -Existing already-issued credentials retain their normal expiry behavior. +The policy-schema migration alone does not change credential expiry. The +v0.4.0 credential cutover does: clients must reacquire credentials and replace +Spaces DPoP with HTTP message signatures. OAuth DPoP is unchanged. Before deploying, take a consistent database backup and volume snapshot, rehearse startup on a disposable copy, and compare the account, policy, membership, and diff --git a/docs/space-host-migration.md b/docs/space-host-migration.md index a6c6f06..4517ec2 100644 --- a/docs/space-host-migration.md +++ b/docs/space-host-migration.md @@ -74,14 +74,17 @@ the DID endpoint automatically. In ZDS, the authority-side state is currently spread across these concepts: - space type and key -- simple-space policy, managing app, and app-access configuration -- simple-space member list -- writer registry with each repo's latest revision and hash +- independent read/write policies, managing apps, and app-access configuration +- simple-space member read/write grants +- writer registry with each repo's latest revision, hash, and space revision +- the authority's latest sequence checkpoint - notification registrations - authority signing-key custody through the resident account Writer record blocks and blobs are not space-host state and must not be copied -as part of a host-only move. +as part of a host-only move. Repo hosts separately own the persistent revocation +cache and pending writer-notification outbox; those follow repo-host state, not +just the authority configuration. The proposal intentionally permits custom host policy. Consequently, a generic destination cannot promise to reproduce an arbitrary source host's policy. diff --git a/docs/testing.md b/docs/testing.md index c75247f..6a30478 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -1,9 +1,11 @@ # Tests and CI -Tangled runs `.tangled/workflows/ci.yml` for branch pushes, pull requests, and +The configured Tangled workflow, `.tangled/workflows/ci.yml`, runs for branch pushes, pull requests, and manual runs. Formatting, unit tests, both HTTP smoke suites, and the ARMv6 ReleaseSafe build must all pass before a push to `main` deploys to Fly. -Pull requests, other branches, and manual runs never deploy. +Pull requests, other branches, and manual workflow runs never deploy. Check the +[deployment observation](deployment.md#october-1-deployment-observation) if a +push does not start a pipeline. ## Local checks @@ -14,8 +16,8 @@ just smoke-all git diff --check ``` -`just smoke-all` builds the PDS and DPoP helper once, then runs the public and -permissioned suites concurrently. Each suite gets its own temporary database, +`just smoke-all` builds the PDS and Spaces HTTP-signature helper once, then runs +the public and permissioned suites concurrently. Each suite gets its own temporary database, blobstore, and response files. Default ports are 2585 (public PDS), 2586 (mock PLC), and 2587 (permissioned PDS). Both suites must succeed. Failed runs retain diagnostic files and print their directory; successful runs remove them. @@ -26,6 +28,10 @@ the binaries they need. The standalone permissioned suite defaults to 2586. Use `ZDS_SMOKE_PORT`, `ZDS_SMOKE_PLC_PORT`, and `ZDS_PERMISSIONED_SMOKE_PORT` to override the ports. +The optional [October 1 SDK interop lane](permissioned-data.md#october-1-upgrade) +checks signatures in both directions. The SDK installation is external to the +repo; the default smoke suites do not require Node or npm. + CI starts cold on every run. The workflow declares a `cache` entry for `.ci-cache` (compiler and package cache, keyed on `build.zig.zon` and `tools/ci-zig.sh`), and spindle.tangled.sh accepts the field, but as of