An AT Protocol Personal Data Server written in JavaScript pdsjs.dev
pds atproto
README.md

@pdsjs/appview #

The primitives a small AppView is made of: the live tail, the archive it catches up from, the index they fold into, and the historical pass that fills it. A service composes them with its own collections and serves its queries from the store. The git-host package is the first consumer.

  • jetstream — a Jetstream v2 consumer. JSON envelopes on one websocket, filtered by collection server-side, the cursor resumable by sequence, a refused cursor handed back to the caller to resume from the archive. Events map onto three shapes: a record write, a record delete, and a handle update; deleted accounts come as account events the fold applies.
  • archive — the v2 network replay. planSnapshot pages the archive into segment files or block ranges, getBlock and getSegment stream them, and the block decoder reads the columnar rows straight off disk. The same folding the live tail uses converges the replay to network truth. On a metered instance the HTTP endpoints take a Bearer API key; a 429 names a Retry-After the client honors.
  • store — the SQLite index. appview_record holds each record whole as JSON; appview_link is the reverse index, one row per at-uri a record names, keyed by the JSON path it names it at. Both commit together, so a read never sees half a write.
  • discovery — the DiscoveryPort over the link table: which records, in which accounts, name this subject at this path. The same question Constellation answers, from an index the service holds itself.
  • backfill — the first-boot walk, for a Jetstream without an archive to replay. com.atproto.sync.listReposByCollection names every account holding a watched collection; each account's PDS is read from its DID document and each collection is listed into the store. Resumes from the stored pagination cursor.
  • watch — the composition: the archive replayed when a stored cursor is refused or absent, jetstream events folded into the store, the cursor checkpointed on a timer, at-least-once by design.

The store takes any driver that speaks the @pdsjs/storage-sqlite/driver interface, which better-sqlite3 and node:sqlite both implement natively. The consumer is Workers-clean: no node imports. The archive's zstd frames ride the same injected decompressor the live tail uses.