zat.dev / zds
atproto pds in zig pds.zat.dev
pds atproto
Star
Fork 1
Watch
zds docs permissioned-data.md
26 kB
Markdown
at fix/default-appview-proxy

permissioned data #

Permissioned data support is experimental and operator gated. Treat the current ZDS surface as an alpha implementation that can change incompatibly, not as a claim of complete parity with the evolving upstream proposal. ZDS documents the com.atproto.space.* routes as experimental, and only enables the handlers when explicitly configured:

ZDS_PERMISSIONED_DATA=true

When the flag is off, permissioned-data routes return a structured MethodNotImplemented response and do not create or mutate permissioned-space rows. The API reference marks the space routes with x-zds-experimental: true so clients can treat the namespace as experimental even when an operator has not enabled it.

The OAuth server follows the same flag. When it is off, a PAR or token request that asks for a space: scope, or for an include: permission set that expands to one, is rejected with invalid_scope. Clients can therefore treat the presence of a space: grant on an issued token as proof that the PDS serves the space routes: ZDS never hands out a grant it will answer with MethodNotImplemented.

references #

The upstream design is still moving. The May branch included protocol-level member lists. The later discussion moved toward making space credentials the protocol substrate and leaving reader/group semantics to applications or space-host policy. Proposal PR #94 keeps that broad direction and defines a baseline com.atproto.simplespace.* management surface for PDS-managed spaces. ZDS targets the October 1, 2026 alpha at atproto commit 679724ad62eb9a02f7f40429c3c4fdd65d3e4c79 and SDK snapshot 0.0.0-spaces-alpha-20261001173819. The migration reference covers this release. Use its pinned Lexicons and handlers for wire contracts; the moving branch can contain later changes.

Feedback should respect the research behind the sketch and avoid overfitting to ZDS or plyr.fm. Name concrete implementation pressure, but frame it as input for the broader protocol shape.

The current local motivation is private data storage on a resident's PDS. That is narrower than proving every group/community use case. Recent proposal discussion reinforces the same posture: groups are the hard case that may justify the primitive, but ZDS should not bake group semantics into the PDS just to get private records and blobs working early.

current prototype surface #

ZDS splits protocol data routes from baseline PDS management routes:

  • com.atproto.space.*: getSpace, listSpaces, listRepos, getDelegationToken, getSpaceCredential, records, blobs, signed commits, full and incremental repo sync, blob enumeration, notification registration and removal, write notifications, credential revocation, and deletion notifications
  • com.atproto.simplespace.*: createSpace, getSpace, updateSpace, deleteSpace, putMember, removeMember, listMembers, and the managing-app checkUserAccess hook

ZDS uses the proposal names for the credential and sync-read surface: getDelegationToken and listRepoOps.

createSpace anchors the space on the authenticated DID and takes {spaceType, skey?, readPolicy, writePolicy, appAccess}. Both policies use the com.atproto.simplespace.defs unions #publicPolicy, #memberListPolicy, or #managingAppPolicy{managingApp}; appAccess is #open or #allowList{allowed}. getSpace returns both policies. updateSpace replaces only supplied fields. putMember upserts required read and write booleans. The old policy field is rejected and addMember is removed. See the client and storage migration guide.

listRepoOps pages with an opaque rev/idx cursor: a full page (limit, default 100, max 1000) carries cursor and no commit; the page that reaches the head of the oplog carries the signed commit instead. listRepos uses the same default and maximum, but orders latest writer states by spaceRev. Each entry contains did, repoRev, hash, and spaceRev. Every nonempty page returns its last spaceRev as cursor; an empty page omits it. Cursors are opaque strings and exclusive checkpoints. A writer can reappear during pagination when it advances; consumers must process it idempotently.

ZDS deliberately does not expose the older protocol member-list routes or the pre-proposal lifecycle names:

  • getMembers
  • getMemberState
  • getMemberOplog
  • notifyMembership
  • com.atproto.space.createSpace
  • com.atproto.space.updateSpaceConfig
  • com.atproto.space.deleteSpace

com.atproto.space.getSpace is the pre-alpha name for com.atproto.simplespace.getSpace; both answer with the same body.

getSpace and listSpaces do not return members or isMember.

getSpace accepts the proposal's space credential when called by a syncer. ZDS also retains resident OAuth/session access for its account UI; that is an additive local access path, not a substitute for credential-authenticated host interoperability.

access model #

ZDS treats permissioned data as a space credential and writer-repo substrate, not as a universal group-membership system.

Applications own rich reader/group semantics: label rosters, supporter access, personal private libraries, community roles, follower-only access, and similar policy all belong above the low-level PDS substrate. If an application needs portable access state, it should model that state as application records in a permissioned space rather than relying on a universal PDS-wide member list.

Cross-space role inheritance is the same kind of application policy. Habitat's Chalk editor (September 2026) puts a document and its comments in two spaces and derives comment access from document access inside its own space host, then asks the community to standardize those relations. Nothing in that shape needs a PDS primitive: a managing-app authority answers checkUserAccess for the comments space by consulting the document relation itself. The one substrate limit to keep in mind is that checkUserAccess returns a single authorized bit, so a viewer/commenter/editor split cannot be enforced by a ZDS-hosted simplespace alone; an app that needs tiered write access hosts the space itself or splits tiers across spaces, as Habitat does.

simplespace membership is stored only as baseline space-management policy state. It is not exposed as a protocol sync surface. For a member-list space, ZDS checks the member's read flag for credentials and write flag for writer authorization. A resident may store records in their own repo even when the authority denies write access; the authority then excludes that update from writer registration and notification forwarding. Each policy is independent. For public, any requester can be authorized. For managing-app, the authority calls the configured service's com.atproto.simplespace.checkUserAccess method with access=read or access=write and service auth. Write checks omit clientId. Resolution, transport, and response errors deny access. The generic PDS implementation of that method returns authorized: false; policy services replace it with their own application-layer decision.

readPolicy, writePolicy, and appAccess use the lexicon-union wire shape on every simplespace route: {"$type":"com.atproto.simplespace.defs#memberListPolicy"}, {"$type":"com.atproto.simplespace.defs#managingAppPolicy","managingApp":"did:…#svc"}, {"$type":"com.atproto.simplespace.defs#open"}, or {"$type":"com.atproto.simplespace.defs#allowList","allowed":[...]}. A managingApp must be a DID with an optional service fragment; anything else is UnsupportedPolicy. Delegation tokens identify the user, not the app. For allow-list spaces, ZDS verifies the separately supplied clientAttestation against the client's published metadata and JWKS before using its client_id. Credential issuance requires an HTTP Message Signature covering authorization, with a P-256 did:key in keyid. The credential binds that key in cnf.kid and, when attested, records client_id. Credential requests use Authorization: Atproto-Space ... and sign authorization plus atproto-space-audience. The audience is the repo owner's DID for repo reads and the authority's bare DID for space-host operations. Signatures are reusable for the same credential and audience; OAuth DPoP and single-use delegation tokens are unchanged.

Credentials default to 600 seconds and may last at most 3600 seconds, with five seconds of clock skew. notifyCredentialRevoked accepts 1–100 credential JWT IDs, authenticated by the exact authority and addressed to a local repo DID. Revocations persist for 3610 seconds from receipt and are enforced on credential use. Removing a member prevents new issuance; it does not currently send revocations automatically.

Writer notifications are keyed by writer repo. notifyWrite verifies service auth from the writer repo to the authority's #atproto_space_host audience (the bare authority DID is also accepted). It records repoRev and the SHA-256 digest of the writer's LtHash state, atomically assigning a strictly increasing space-wide spaceRev. Duplicate/older repo revisions are ignored, and revisions more than five minutes in the future are rejected. Forwarded notifications include spaceRev and, after the first update, prevSpaceRev. listRepos is space-credential-only and returns this authority-maintained writer set; it never exposes the raw LtHash state or a reader/member list.

registerNotify stores a 24-hour whole-space subscription for a service DID, resolving either the named service fragment or the DID's PDS endpoint. unregisterNotify removes that subscription idempotently. Repo writes save pending notifications in the same SQLite transaction as the records and commit state. The delivery worker coalesces newer state per writer/space, uses expiring leases, and retries transient failures with backoff for up to 24 hours per new revision. Permanent errors stop delivery. Authority-to-syncer fanout remains best effort: syncers recover gaps through listRepos from their last safely processed checkpoint, then fetch data from the affected repo hosts.

When notifying a remote authority, ZDS resolves its dedicated #atproto_space_host service when present and falls back to #atproto_pds. This preserves the proposal's separation between an authority's identity, space host, and account PDS.

client-visible contract (verified across implementations, 2026-08-21) #

This historical comparison informed plyr.fm (#1897, #1898). The behaviours were checked against the reference PDS (permissioned-data branch), rsky, and atproto-crates so that nobody mistakes them for ZDS quirks and "cleans them up".

behaviour reference rsky atproto-crates ZDS
scopes_supported lists space:* (or any granular scope) no (atproto + transition:* only; "dynamic scopes can't be enumerated") no no no (it did until 2026-08-21; the hints were a ZDS invention nobody consumed)
authority=* in a space: grant matches any authority; self resolves to the granting user at token issuance yes yes yes yes
getDelegationToken needs only a covering read grant, signs with the requester's own repo key, and never checks that the authority is local yes yes yes yes
simplespace.listMembers is the authority's private state owner-only (read_self scope + owner assert) owner-only (manage=update) any member with read_self owner-only (manage=update)

Clients must not infer capability from scopes_supported: the grant on the issued token is the only portable signal. listMembers is the one row where implementations differ; a client should treat it as owner-only.

The September policy update supersedes the ZDS listMembers scope cell above: ZDS now requires the owner and a covering read grant. Other implementations in this historical table have not been rechecked for the October alpha.

storage shape #

ZDS stores many actor repos in one SQLite database, so permissioned data uses explicit space-scoped tables instead of the public repo tables:

  • permissioned_spaces: canonical authority-space config keyed by space URI
  • permissioned_space_actor_state: actor-local authority/deleted state keyed by (space, actor_did)
  • simplespace_members: baseline PDS-managed member-list policy state keyed by (space, member_did)
  • permissioned_space_records: current records keyed by (space, repo_did, collection, rkey) with CID, AT JSON value, repo revision, and indexed timestamp; canonical DAG-CBOR is reconstructed and CID-checked for full-state export
  • permissioned_space_repos: repo-host state, including the full LtHash state needed to update and sign that writer's local permissioned repo
  • permissioned_space_writers: authority-host writer registry, containing only each known writer's latest repo revision, space revision, and 32-byte commit digest
  • permissioned_space_record_oplog: incremental record changes by (space, repo_did, rev, idx)
  • permissioned_space_notify_registrations: expiring service-identified write-notification subscriptions
  • permissioned_space_sequence: persistent last revision for each authority space
  • permissioned_space_notify_outbox: durable, coalesced repo-host notifications
  • permissioned_space_revocations: expiring (space, jti) revocations
  • permissioned_space_used_delegations: consumed one-use delegation-token IDs
  • permissioned_space_used_client_attestations: consumed one-use client attestation IDs

Space roots use the canonical proposal URI:

at://{authorityDid}/space/{spaceType}/{skey}

Permissioned record URIs append {authorDid}/{collection}/{rkey}. ZDS migrates the earlier experimental URI scheme in storage and does not accept it at the HTTP boundary.

listSpaces enumerates actor-local space rows: spaces the resident owns or has materialized a permissioned writer repo in. Being named in a simplespace member list does not make a space appear there.

OAuth space grants use authority=, repeated action= record operations, and separate repeated manage= space-management operations. authority=self is resolved to the granting account's DID when the token is issued.

OAuth reads are limited to the authenticated account's own permissioned repo. Whole-space writer enumeration and cross-repo reads require a space credential. Delegation tokens and client attestations are short-lived and consumed together in one transaction when the authority mints that credential. A missing or invalid issuance signature does not consume either token. Reusing a consumed delegation token still fails even with a valid signature.

Permissioned records and blobs are not public repo records. They must not be squeezed into records, repo_blocks, commits, or seq_events.

host roles and blob ownership #

The proposal separates three roles that are often served by one PDS:

  • the PDS authenticates its resident and accepts writes
  • the repo host stores and serves that resident's permissioned repo and blobs
  • the space host describes the space, decides whether to mint space credentials, lists known writers, and routes notifications

The roles may be implemented by different services, but that does not imply centralizing every writer's records at the space host. In the proposal's normal topology, each writer's permissioned repo and referenced blobs remain on that writer's repo host. A syncer obtains a space credential from the space host, discovers writers there, and then pulls records and blobs from each writer's repo host.

ZDS currently serves all three roles for its resident-owned simplespaces. In the plyr.fm path, com.atproto.repo.uploadBlob places bytes in the resident's blobstore and a subsequent permissioned record write associates the CID with that exact space. Public com.atproto.sync.getBlob remains unavailable unless a public repo record also references the blob. Permissioned com.atproto.space.getBlob requires both valid read authority and an exact (space, repo, CID) record reference; a credential for one space cannot be used to retrieve a blob referenced only by another.

A separate space host may choose to store permissioned records itself as a compatibility layer for PDSes that do not implement permissioned repos. That is a valid experimental deployment shape, but it is not equivalent to the proposal topology: the writer's PDS cannot infer the remote private record's blob reference, so retention and garbage collection require an additional contract. ZDS should not invent that contract locally before interoperability testing establishes the shared requirement.

Before cross-implementation testing, verify these invariants independently:

  • a previously uploaded blob becomes reachable only after a record in the requested space references it
  • the same CID remains unavailable through another space
  • public blob reads remain unavailable for private-only references
  • record update and deletion update the space-scoped blob reference index
  • old permissioned records are backfilled into that index before stricter read authorization is enabled

The draft proposal and its reference branch remain works in progress. Treat their stated topology and wire contracts as the interoperability target while testing security-sensitive behavior rather than inheriting implementation gaps.

com.atproto.space.listRecords returns current record values by default, in line with the permissioned-data proposal. Consumers that only need the collection, rkey, and CID listing can pass excludeValues=true to avoid materializing record JSON.

com.atproto.space.listRepoOps follows the same value rule. Create and update operations inline a value only while that operation's CID is still the current record CID; deletes and superseded operations remain metadata-only.

com.atproto.space.getRepo is the full-state recovery path. Its CAR declares the freshly generated deniable signed commit and a DAG-CBOR record index as its two roots. The record index uses canonical DAG-CBOR map ordering (key length, then bytewise key order), and the commit MAC uses the proposal's direct HKDF-Expand construction. Blobs are deliberately excluded; listBlobs enumerates referenced CIDs and getBlob serves the authorized bytes.

The commit signature authenticates only (space, author, rev, ikm), not the record-set hash. Anyone holding a commit can replace its records and recompute the hash and MAC without the author's signing key. verifyRepoCarFull checks the signed context and CAR consistency; callers must establish content provenance separately, such as through an authenticated fetch from the author's resolved repo host. Forwarded exports do not provide third-party evidence of record authorship. This deniability is intentional in proposal 0016.

LtHash also does not provide a compact record inclusion proof: subtracting an arbitrary record's contribution and adding it back works even when the record was never in the set. getRecord returns a URI, CID, and value, not an inclusion proof. A secondary service serving selected records therefore needs its own trust relationship with consumers or independent verification against the source; it cannot use the repo commit as transferable proof of those records.

Zat first #

Before implementing local primitives, check Zat's current public API. ZDS uses zat for syntax, TIDs, DID/handle resolution, JWT helpers, DAG-CBOR, CAR, MST, repo verification, OAuth helpers, JSON helpers, and streaming clients.

Likely future Zat candidates:

  • SpaceUri parsing/formatting
  • LtHash and set commitment primitives
  • proposal-shaped delegation-token and space-credential JWT helpers

Do not edit or patch the sibling zat repo from ZDS without stopping and making the proposed Zat change explicit.

code boundaries #

Permissioned data is experimental code and should stay segmented:

  • HTTP dispatch and XRPC semantics live in src/atproto/space.zig.
  • Experimental protocol helpers live in src/internal/permissioned_data.zig.
  • The resident private-record browser lives in the /account/spaces section rendered by src/internal/account.zig.
  • OAuth scope parsing for space: scopes lives in src/internal/scopes.zig.
  • Persistent state lives in permissioned-data tables in src/storage/store.zig.
  • API-reference surfacing is marked experimental through the space router group and x-zds-experimental.

Normal repo writes, sync events, account auth, and appview proxy behavior should not learn permissioned-data special cases unless the protocol requires a shared primitive. Prefer a permissioned-data helper over scattering conditionals into stable PDS code.

October 1 upgrade #

Startup creates the revocation, sequence, and notification tables and backfills existing writer rows with ordered space revisions. Existing records, blobs, policies, and memberships retain their format. Old DID-based writer cursors must be discarded; clients bootstrap again from listRepos without a cursor. Clients must reacquire credentials and use HTTP signatures. The old Spaces DPoP scheme and type request field are not compatibility paths.

The local consumers requiring follow-up are plyr.fm's Python client, Doodl's browser/live/preview services, and Lore's extension and website. Doodl and Lore also need the prior release's policy split and all three still have membership callers to update. These are the downstream migration requirements for v0.4.0; ZDS does not translate the previous wire contract for old clients.

The optional SDK interop lane runs against an externally installed October 1 @atproto/space snapshot:

ZDS_SPACE_SDK_DIR=/path/to/sdk-test-project just smoke-permissioned

That project must contain node_modules/@atproto/space and its dependencies. Pin @atproto/lex-cbor and @atproto/lex-data to the same snapshot if npm resolves their prerelease ranges to the incompatible stable 0.0.0 packages.

checks #

Run both smoke lanes when touching this area:

just test
just smoke
just smoke-permissioned
git diff --check
zig zen

October 1 release verification #

The implementation follows the October 1 migration prompt and pinned reference implementation described above. HTTP signature parsing remains local to ZDS; DID-key decoding and JOSE verification use zat. The published October 1 SDK passes bidirectional signature interoperability checks, including unpadded base64, reusable proofs, high-S signatures, and repository audience isolation.

Five alternating local ReleaseFast storage runs against the preceding committed HEAD, with 1,000 seeded records, showed median throughput changes of +2.5% for writes, +0.6% for single-record reads, -2.9% for record listings, -1.5% for repo exports, and -4.7% for writer listings. These are synthetic storage measurements. A separate ReleaseSafe HTTP probe completed 4,000 credential reads without failure; at 16 concurrent callers it measured about 5,800 requests/second and 4.3 ms p95 latency. It is not a sustained production soak.

Cross-PDS performance comparison remains a follow-up: the existing official PDS and Tranquil benchmark probes cover public repos, not equivalent Spaces work. The SDK interop lane establishes wire compatibility, not performance parity. The current CAR/commit format remains intentionally unchanged until the future alpha format migration; downstream application migrations are separate work.