You point PDSuite at a PDS. It runs a set of checks against the server and prints a report. Each check says if the server follows the AT Protocol specification, the lexicon contracts, or the behavior of the reference implementation.

The tool is a command-line program. It is also a library, so you can run it in CI.

What a check is #

A check is a small, named test. Each check has these parts:

An id, for example repo.commit.rev-monotonic. The id never changes.
An authority. This is the source of the rule. See the table that follows.
A set of expectations. These are the assertions the check makes.
A rationale. This says why the rule matters for the network.

The authority tells you where the rule comes from:

Authority	Source	A failure means
`spec`	The written specification at atproto.com	The server violates the protocol
`lexicon`	The pinned `com.atproto.*` lexicon files	The server violates its declared contract
`interop`	The `atproto-interop-tests` vectors	The server fails a cross-implementation fixture
`reference`	The observed behavior of the reference PDS	The server differs from what clients are tested against

The specification has gaps. Some behaviors are not written down. For these, the tool compares the target against the reference PDS. A difference from the reference is a divergence. A divergence is a warning, not a failure. It tells you where a client can behave in a different way than you expect.

Verdicts #

Each check ends with one verdict:

Verdict	Meaning
`pass`	All expectations hold
`fail`	A `must` expectation does not hold
`warn`	A `should` or `divergence` expectation does not hold
`skip`	The check does not apply to this server
`error`	The check itself failed, so the result is unknown

A fail is a hard violation. A warn is a softer problem or a known difference. A skip is not a failure. It means the server does not have the feature the check tests, for example an optional endpoint.

Install #

You need Node.js 22 or later and pnpm.

Clone the repository, then install the dependencies:

pnpm install

Usage #

Run a read-only check against a server:

pnpm check https://pds.example.com

Read-only mode is safe. It sends only GET requests. It does not create accounts or write data.

Run the full checks. This mode creates a test account and writes records:

pnpm check https://pds.example.com --mode mutate

CAUTION: Do not use --mode mutate against a server that you do not control. The mode writes records to a test account.

If the server needs an invite code, set the code first:

export PDSUITE_INVITE_CODE=your-code-here
pnpm check https://pds.example.com --mode mutate

Get the report as JSON for CI:

pnpm check https://pds.example.com --json

Integration mode covers relay-proxied reads (sync.listHosts, sync.listReposByCollection). It is a pair of lifecycle-mode checks that need a configured upstream. See docs/integration-mode.md.

Lifecycle mode also covers account state transitions: deactivation, reactivation, and deletion, plus the firehose events and redistribution rules around them. See docs/lifecycle-mode.md.

Run only the checks whose id starts with a prefix:

pnpm check https://pds.example.com --only oauth

Target profiles (declared divergences) #

A server can diverge from the spec on purpose. A target profile lets the server's maintainers declare those divergences, with a note and a link to their own documentation. Load one with --profile:

pnpm check https://pds.example.com --profile reference-pds

A declared divergence is acknowledged, not hidden. The report still shows it, annotated with its note and link. If the divergent behavior is a must failure, the report downgrades it to a warning. Owned and visible, never silently silenced. Profiles live in src/profiles/ and are co-owned with the maintainers of each server. See src/profiles/reference-pds.ts for the format.

Run against a local third-party PDS #

You can run the suite against a real third-party PDS that you control, with no external account. The repo includes a minimal local tranquil-pds stack (postgres + a mock PLC directory + tranquil-pds) at docker/docker-compose.local.yaml.

The stack pulls the prebuilt tranquil-pds image from atcr.io, which needs a one-time login (an ATProto handle + app-password via the docker-credential-atcr helper):

docker-credential-atcr login        # one-time, device flow
docker pull atcr.io/tranquil.farm/tranquil-pds:latest

# Build the mock PLC image and bring up the stack (from the PDSuite repo root)
docker compose -f docker/docker-compose.local.yaml up -d --build

# Run the full suite against it (handles use the stack's .devpds.com domain)
PDSUITE_HANDLE_DOMAIN=devpds.com pnpm check http://localhost:3000 --mode mutate --profile tranquil-pds

# Tear down and wipe all state
docker compose -f docker/docker-compose.local.yaml down -v

The mock PLC is an in-memory, writable PLC directory (the reference @did-plc/server with a mock DB), so account creation registers real did:plc identities that vanish when the stack stops. The SSRF guard of tranquil-pds only allows loopback PLC URLs in a release build. So the PLC shares the network namespace of the PDS container (network_mode: service:pds).

The check tiers #

The checks are in tiers. Each tier tests a different surface of the server:

Tier	Surface	Mode
0	Discovery, OAuth metadata, error envelope	readonly
1	Repository write and read round-trip	mutate
2	The firehose event stream	mutate
3	Authentication flows, including headless OAuth	mutate
4	Comparison against the reference PDS	mutate

Tier 3 drives the full OAuth grant without a browser. It uses the JSON API that sits behind the provider consent page (/oauth/par, /oauth/sign-in, /oauth/consent, /oauth/token). This works on the reference PDS and on third-party servers that expose the same endpoints.

Write a check #

A check is a TypeScript file. It exports one object from the check() function. The example that follows is complete:

import { check, must } from '../types.js'

export default check({
  id: 'xrpc.error.envelope',
  title: 'Unsuccessful responses use the standard error envelope',
  tier: 0,
  mode: 'readonly',
  authority: 'spec',
  specRefs: ['https://atproto.com/specs/xrpc#error-responses'],
  expectations: [
    must('envelope-shape', 'Error responses are JSON with a string `error` field.'),
  ],
  rationale:
    'Clients branch on the `error` name for retries and UX. A server that returns ' +
    'HTML or plain text makes every client error path worse at once.',
  async run(ctx) {
    const res = await ctx.xrpc('com.atproto.sync.getRepo', { params: {} })
    const body = res.body as Record<string, unknown>
    ctx.assert(
      'envelope-shape',
      typeof body?.error === 'string',
      { status: res.status, body },
    )
  },
})

The parts you must write:

id, title, tier, mode, authority, rationale. The rationale is mandatory.
expectations. Use must(), should(), or divergence(). Give each an id and a statement.
run(ctx). This does the interaction and calls ctx.assert().

The ctx.assert() call takes three arguments: the expectation id, a boolean condition, and evidence. The evidence is data that the report shows when the assertion fails. Put the request and the response in the evidence. A failure must show what happened on the wire.

The context object #

The run function gets a ctx argument with these methods:

ctx.xrpc(method, opts) calls an XRPC endpoint on the target.
ctx.fetch(path) makes a raw HTTP request. Use this for paths that are not XRPC endpoints, for example /.well-known/oauth-protected-resource.
ctx.account('alice') gives you a provisioned test account. Declare it in requires.accounts first.
ctx.assert(id, condition, evidence) records an assertion.

Declare what the check needs #

If a check needs an account or an endpoint, say so in requires:

requires: {
  accounts: [{ id: 'alice', caps: ['createRecord'] }],
  endpoints: ['com.atproto.repo.createRecord'],
},

The runner reads requires before it runs the check. If the server does not have the endpoint, the check becomes a skip, not a fail. This keeps the report clean for servers that do less by design.

The contract generator #

Most endpoint checks are not written by hand. The generator reads the pinned lexicon files and writes them.

Run the generator:

pnpm generate

The generator writes one check file per endpoint into src/generated/. These files are machine-written. Do not edit them by hand. When the lexicons change, update the pin and run the generator again. The diff shows what changed in the protocol.

The report #

The report has one line per check. A failing expectation shows its id, its authority, its severity, and its evidence. At the end, the report gives a summary of claims grouped by authority. A pass on spec checks and a pass on reference checks are different claims. The report shows both.

Design principles #

Three rules shape the tool:

Every expectation cites its source. A failure always says where the rule comes from. This is necessary because the specification does not define every behavior.
Skips are not failures. The tool computes what the server can do, then tests only that. It reports a skip with a reason.
A failure shows the wire. Every failure includes the request and the response. You can paste the report into a bug report.

Repository layout #

Path	Contents
`src/types.ts`	The check DSL and the core types
`src/runner.ts`	The check runner, the skip logic, and profile/waiver application
`src/context.ts`	The check context and assertion recording
`src/report.ts`	The terminal and JSON reporters
`src/discover.ts`	The discovery pass (describeServer, OAuth metadata, and the pinned endpoint inventory probe)
`src/checks.ts`	The single check-list assembly point every entry point uses
`src/provision.ts`	The account provisioner (creates test accounts on the target)
`src/firehose.ts`	The firehose consumer harness
`src/oauth.ts`	The headless OAuth grant driver
`src/oracle.ts`	The in-process reference PDS for differential checks
`src/generate.ts`	The contract-check generator
`src/contract-runner.ts`	Executes generated contract-check variants
`src/checks/`	Hand-written checks, by tier
`src/generated/`	Machine-written checks. Do not edit
`src/profiles/`	Co-owned target profiles (declared divergences)
`docker/`	The mock-PLC image for the local third-party-PDS stack
`docs/`	Guides (writing a check, and more)

License #

MIT.

Clone this repository

PDSuite #