The pre-launch test matrix #
Every check this project wants green before anything reaches a real zone, in
the three tiers it runs in: what an agent can break on one machine, what needs
an unadvertised instance on a real zone, and what needs a login in somebody
else's system. Each row names the test or scenario that covers it, and says
none where nothing here does.
Tiers are ordered. Nothing in tier 2 starts until tier 1 is green; nothing in
tier 3 touches an account that is not disposable. This page is the map, not
the gate: the gate is the irreversibles — the DID scheme, the lexicon shapes,
the zone, and what is written into other people's servers.
A none is a row somebody has to do something about, and the two ways to do
that are a test and a tier. Moving a row up a tier is the better one: a check
that needs a real zone is a check that runs once a week, and a check that runs
in this repository runs on every push.
Tier 1 — local, sandboxed, agent-driven #
Everything here runs on one machine with stand-ins for the network. The
command column is what runs it.
The harness itself #
| Check |
Command |
What covers it |
An ecosystem replica in one compose file: the reference PDS, an appview, an indigo relay, goat, an ACME directory and a resolver for the zone |
none |
none |
A local TLS and DNS story a foreign resolver accepts: a local CA the replica trusts, so did:web resolves over HTTPS |
none |
crates/didbot-tls/tests/acme_conformance.rs drives ACME against an in-process directory; nothing yet issues into a replica |
Local TLS: a .localhost zone served from an authority kept per machine, reused across starts, refused for any other zone, and verified by a real client |
cargo test -p didbot-tls |
crates/didbot-tls/tests/local_authority.rs completes a handshake against the leaf a zone serves and shows the authority's name constraint refusing one outside .localhost; src/local.rs covers reuse and the refusal |
| Every binary in one scenario over TLS, against one authority, with each zone's port named |
scripts/scenarios.sh |
crates/didbot/tests/scenarios.rs starts didbot-pds with --tls local and drives didbot, didbot-agentd and a browserless dashboard sign-in over https |
| Sandboxed agent hosts: several containers, each with the plugin and the daemon |
none |
none |
| A red-team brief handed to agents, with the replica's oracles deciding |
none |
none |
Wire and sync #
| Check |
Command |
What covers it |
| A relay inverts every commit kind with no MST inversion failures |
cargo test -p didbot --test conformance |
crates/didbot/tests/conformance/mst_inversion.rs, against the vectors rather than against a relay |
| The stream survives a restart, a snapshot restore, a certificate swap and a reconnect from an old cursor |
cargo test -p didbot-pds --test sequence_restart and cargo test -p didbot-tls --test rotation |
the sequence floor and the certificate swap each have a test; a relay's own view of the two does not |
goat repo export and verify, for every account state |
cargo test -p didbot --test announce_states |
the states and their events; the export is read by this project's own CAR reader in crates/didbot-repo/tests/car_framing.rs |
| An appview indexes a provisioned account: profile, self-label, repo status and account events agreeing |
none |
none |
getRepo with since, listBlobs and getBlob round-tripping through a foreign client |
cargo test -p didbot --test conformance |
crates/didbot/tests/conformance/wire.rs drives them against the vendored documents, not through a foreign client |
Identity #
| Check |
Command |
What covers it |
did:web and handle resolution for an account, the server's apex, and a tombstoned account |
cargo test -p didbot-identity |
crates/didbot-identity/tests/spec_conformance.rs |
| A document rewrite, and what a reader does with signatures made before it |
none |
none |
| Server-minted DIDs: one account per correlation key, a deleted DID never reissued, a caller's label ignored |
cargo test -p didbot-pds --test provisioning |
crates/didbot-pds/tests/provisioning.rs and crates/didbot-pds/tests/naming.rs |
OAuth #
| Check |
Command |
What covers it |
| A full sign-in from a foreign client: push, decision, approval, code, bound token, write, refresh, replay |
cargo test -p didbot-serve --test oauth_account_flow |
the whole flow against the real router, with this project's own client |
Bluesky's own @atproto/oauth-client-node signs in to a running server as a public native client and writes before and after a refresh; as a confidential web client it pushes with its key, is refused approval because a browser would carry its code, and is declined |
npm ci --prefix tools/oauth-interop, then cargo test -p didbot-serve --test oauth_interop |
crates/didbot-serve/tests/oauth_interop.rs runs tools/oauth-interop/sign-in.mjs against didbot-pds and answers each sign-in as didbot-agentd does; without node or the installed tree it passes having said so |
| The loopback client form, and a client whose metadata is served from a hostile origin |
cargo test -p didbot-serve --test oauth_standard_client and --test oauth_interop |
jacquard-oauth's loopback client and @atproto/oauth-client-node's sign in; a hostile origin is not driven |
| Scope narrowing: a token cannot write outside its grant, a tightened ceiling narrows the next write |
cargo test -p didbot-serve --test token_scope |
crates/didbot-serve/tests/token_scope.rs and crates/didbot-scope/tests/ceiling_boundary.rs |
Every scope form the permission spec defines reads without a warning; each other form reads as its warning says, and PAR logs it at warn with the client; a never-granted scope is dropped and Leaflet and Tangled sign in |
cargo test -p didbot-scope, then cargo test -p didbot-serve --test scope_warnings --test permission_sets --test token_scope |
crates/didbot-scope/src/tests.rs, and the three didbot-serve suites against the real router |
| A scripted agent that approves everything, against the default ceiling |
none |
none |
| A second agent cannot redeem a consent reference it did not receive |
cargo test -p didbot-serve --test oauth_account_flow |
the account check is the credential's, and the flow test drives it from both sides |
Every bot.did.* route answers in the shape its document declares |
cargo test -p didbot --test conformance bot_did |
crates/didbot/tests/conformance/bot_did.rs |
Policy and the e-stop #
| Check |
Command |
What covers it |
| The operator's repository unreachable at boot, mid-run, and answering empty: no denial is lost |
cargo test -p didbot-serve --test policy_poll_restart |
crates/didbot-serve/tests/policy_poll_restart.rs restarts with the operator's repository gone, and the kept set refuses before any poll runs |
| One malformed record beside a valid new denial: the valid one binds |
cargo test -p didbot-policy-records |
crates/didbot-policy-records/tests/last_good_across_builds.rs covers a revision a later build will not honour beside the one it keeps enforcing; the pairing with a valid new record at another key is not driven |
| Policy change latency end to end, measured and published as the poll interval |
none |
none |
--pause and --revoke under load: no write lands after the throw, and a restart keeps the stop |
cargo test -p didbot-pds --test freeze_race and cargo test -p didbot-pds --test estop_restart |
writes queued behind a judgment when the stop is thrown, and the latch across a restart; not under swarm load |
| An ownership lapse: the record deleted, the grace window passed, the self-pause, the record restored |
cargo test -p didbot --test handshake |
crates/didbot/tests/handshake.rs, in one process against a stand-in repository |
| The dashboard against real servers, in a browser |
DIDBOT_E2E=1 scripts/test-policy-e2e.sh |
a server, the operator's stand-in PDS, the built site and headless Chrome |
Attestation #
| Check |
Command |
What covers it |
| A replayed claim inside the window, after a restart, and after the account it minted was erased |
cargo test -p didbot-serve --test proof_restart |
the spent-proof window across a restart |
| A copied node key on a second container, and the honest limit that draws |
none |
docs/trust-model.md states the limit; nothing exercises it |
| Clock skew of ten minutes either way: the refusal names skew |
none |
none |
| A node key retired on a running server: the next claim refused, existing contexts locked |
none |
none |
Agent hosts #
| Check |
Command |
What covers it |
| Real sessions in containers: session start, a fan-out of subagents, and every record authored by the context the hook fired for |
none |
none |
| A daemon restart mid-session, a hook race at startup, a daemon unreachable, and a socket only its owner's uid can reach |
cargo test -p didbot-agentd |
the command line and the configuration; the socket's permissions and the races are not driven |
| A hostile in-process subagent acting as its parent |
none |
docs/trust-model.md states the custody limit |
| The dispatcher on a host holding only the agent install: the operator verbs are absent and say so |
cargo test -p didbot-dispatch |
crates/didbot-dispatch/tests/dispatch.rs |
Durability, restart and restore #
| Check |
Command |
What covers it |
| A kill at every sync boundary under load: every acknowledged write present, no unacknowledged one |
cargo test -p didbot-pds --test durability and cargo test -p didbot-pds --test record_heap |
the crash-and-reopen cases, and a power cut after another store's sync of the shared journal; not thousands of them under swarm load |
| A restore from a directory copied mid-load: the sequence floor, the relay's view, the blob check |
cargo test -p didbot-pds --test restore and cargo test -p didbot-serve --test restore_drill |
the store half and the deployment half |
| Compaction, boot time as a function of history, and the refusal at the log budget |
cargo test -p didbot-pds --test write_scale -- --ignored |
the benchmark prints what a write costs as history grows; the boot time and the budget's refusal are not driven |
| Pending sign-ins across a restart |
none |
none |
| A binary opening a directory another binary wrote |
cargo test -p didbot-pds --test upgrade |
crates/didbot-pds/tests/upgrade.rs |
Abuse and load #
| Check |
Command |
What covers it |
| A create flood and a push flood from many addresses |
cargo test -p didbot-swarm --test decision_bounds |
the decision store's bounds under a real swarm; the address-keyed budgets are unit-tested in crates/didbot-serve/src/rate_limit.rs |
| A blob flood to the quota |
cargo test -p didbot-pds --test blob_storage |
crates/didbot-pds/tests/blob_storage.rs drives the upload path against the quota |
| A subscriber that stops reading, with others still on the stream |
cargo test -p didbot --test subscribe_backpressure |
crates/didbot/tests/subscribe_backpressure.rs |
| A swarm at a thousand and at ten thousand accounts: memory per account, restart time, the account wall |
cargo test -p didbot-pds --test store_cost |
the per-account and per-record cost; the restart time and the wall at ten thousand are not driven |
| A swarm behind one address against the per-address budgets |
none |
none |
Operations #
| Check |
Command |
What covers it |
| An image upgrade against a data directory, a rollback by restore, and the crash-loop brake |
cargo test -p didbot-pds --test upgrade |
the directory half; the brake and the rollback are docs/operations.md's runbook |
| Every boot refusal in the operations table, produced on purpose |
cargo test -p didbot-pds --test upgrade, cargo test -p didbot-pds --test second_writer and cargo test -p didbot-serve --test startup_refusal |
upgrade.rs produces the three stamp refusals and second_writer.rs the lock; startup_refusal.rs refuses --tls acme under .localhost before the data directory is claimed |
| A graceful stop on the signal a deployment sends |
cargo test -p didbot-serve --test graceful_shutdown |
crates/didbot-serve/tests/graceful_shutdown.rs |
| The ownership scenarios with the real binaries |
scripts/scenarios.sh |
crates/didbot/tests/scenarios.rs |
Vibescrobble #
| Check |
Command |
What covers it |
| The index reads a relay's stream and survives a relay and a server restart |
none |
none |
| A scrobble written by a real containerised session appears on the canvas with the right operator and parent |
none |
none |
Tier 2 — an unadvertised instance on a real zone #
A real zone, a real certificate, no announcement, on a zone whose every
account will be tombstoned. Nothing here runs from this repository: each row
is an operator following docs/deployment.md and recording what happened in
docs/conformance.md.
| Check |
Where it runs |
What covers it |
tofu apply from an empty state, the delegation, the wildcard and apex records, and staging issuance before production |
the zone |
crates/didbot-dns/tests/route53_zone.rs and crates/didbot-tls/tests/dns01_zone.rs drive the record writes against a hosted zone that answers like the real one |
| The public checkers: pdscheck, the debug tools, the directories, the TLS and header graders |
the instance, from outside |
none |
| Certificate transparency: a monitor subscribed, our own issuance seen and nothing else |
the zone |
none |
| Our own relay in another region crawling over the internet |
the instance, from outside |
none |
| Agent hosts across the internet: hook latency, a daemon behind NAT, real clock skew |
a laptop and the instance |
none |
| A backup and restore drill on real volumes, an image swap, a disk fill, and break-glass reaching the e-stop |
the instance |
crates/didbot-pds/tests/restore.rs drills the store half locally |
| A week of background scanning on 443 |
the instance |
none |
| Vibescrobble deployed against the instance |
the instance |
none |
| The whole zone tombstoned, and what a relay and an appview do with it |
the zone |
none |
Tier 3 — an external login or an external operator #
Everything here leaves a record in somebody else's system, so every row uses a
throwaway account.
| Check |
What it needs |
What covers it |
didbot operate and the policy site writing against a real personal data server |
an account on that server |
crates/didbot/tests/scenarios.rs drives the command against a stand-in repository |
bot.did.* permission sets expanded by somebody else's authorization server |
the published schema records, and that server |
only an external authorization server can answer this |
| An announcement to the network from the disposable zone: admission, the per-host cap, an account visible in a client, and an account signing in to a third-party site |
the network |
none |
| A preference record read against real accounts that publish one |
those accounts |
none |
| A report filed against a disposable account, and what reaches the operator |
somebody else's moderation service |
none |
| What a relay and an appview do when the disposable zone is tombstoned |
the network |
none |
