id: adversarial title: Integration tests at the seams, not inside the components that already pass status: open crates: [didbot-serve, didbot-pds, didbot-identity, didbot-policy, didbot-policy-source, didbot-dns, didbot-tls] dependsOn: [] exitCriterion: > Adversarial integration tests exist for zone containment and DPoP binding, each exercising a request that crosses at least two components, and each fails loudly if either component's assumption about the other quietly changes. #

adversarial #

CLAUDE.md says integration tests exercising multiple system invariants are the most valuable kind. These targets already have unit tests, and unit tests are not what is missing: what is missing is a test at the seam between the components each one assumes will hold up its end.

• Zone containment — narrower than it once was, now that a zone need not sit at or below the PDS's own hostname (see didbot_identity::Zone's docs) — has unit coverage in crates/didbot-identity/tests/identity.rs (hostname_is_at_or_below_respects_label_boundaries, hostname_is_at_or_below_is_case_insensitive) and in vibescrobble.com's index crate (a_suffix_is_not_containment, a_handle_suffix_is_not_containment_either). Each tests containment as a standalone function against a string. Neither drives a request through a server that resolves a hostname, decides whether to serve it, and decides whether to accept a vouch about it, using the same containment check at each step.
• DPoP binding has unit coverage in crates/didbot-serve/src/oauth/dpop_seam.rs, including a replay window, a thumbprint mismatch, and a verifier that refuses by default. Each test constructs a proof and calls the verifier directly. None of them goes through authorize.rs, token.rs and a resource request in sequence — the seam between "a token was minted bound to key A" and "a resource server later checks a proof against key A" is the one nothing exercises end to end.

Zone containment: the seam between resolution, serving and vouching #

The containment left to test is not "is this zone under the PDS's own hostname" — that rule is gone — but two narrower ones: a requested handle must sit at or below the zone the deployment mints under, and the index crate must not credit a server with an agent whose DID sits outside what a vouch actually covers. A suffix sharing no label boundary (agents.localhost.attacker.com against agents.localhost) is the shape hostname_is_at_or_below_respects_label_boundaries and a_suffix_is_not_containment already refuse in isolation, and is what the handle-forgery tests in crates/didbot/tests/zone_containment.rs drive through the real HTTP surface.

• The two containment checks agree. didbot-identity and vibescrobble.com's index each implement containment against the rule described in agent-accounts.md's Done list — "compared label by label" — and nothing confirms they compute the same answer for the same input. Now that the index is a separate repository consuming didbot-identity as a git dependency, this seam is that repository's to close: a test that feeds one adversarial hostname through both and asserts agreement, because two correct implementations that happen to disagree on one input is exactly the failure mode neither component's own unit tests can see.

DPoP binding: the seam between mint and verify #

Degenerate values: the seam between a configuration path and a comparison #

The suites above test seams between components. This one tests the seam between a component and the configuration that feeds it — the class where a comparison is correct for every value except the one a failed configuration path produces, and nothing in the codebase refuses to produce that value.

Two live examples, both found by reading rather than by a test, are written up in the Done list below. What they share is the shape: a check that reads as obviously correct, a degenerate input nobody would type, and a path that supplies the degenerate input on its own without anybody typing it. The tests that missed them missed them the same way — each asserted a handful of specific values rather than the property.

Where the remaining risk is, in rough order:

• Route53Dns::resync runs at provider construction. It exists so a fresh process does not start from "nothing published" against a zone that already holds accounts. Its two callers are both in crates/didbot-serve/src/bin/didbot-pds.rs, both at construction, so the bookkeeping it establishes is the zone as of startup and holds that shape for the life of the process. A name written to or removed from the zone afterwards is still answered from that read: withdraw answers NotPublished from it without making an API call, so a hostname deprovisioned elsewhere keeps resolving. publish is safe (Route53 refuses the colliding CREATE), so this is a leak rather than a takeover. What closes this item is a pass that revisits the zone while the process runs.
• Route53 error classification is a substring match on the message text (lower.contains("not found")). A backend misconfiguration whose message happens to contain those words is reported to the caller as a definitive "that record is not there". Fail-closed; the cost is a misdiagnosis.
• NameRegistry::claim's journal failure reads as "the name is taken". A full or read-only disk makes every retry fail identically, and issue reports name exhaustion — so an operator is told this deployment has run out of names when what it has run out of is disk.
• MimePattern::intersect is incomplete. image/* and */png overlap at image/png, and it answers None, because it only succeeds when one pattern contains the other. Fail-closed, and unreachable until a ceiling actually uses a half-wildcard, but it is the one place in the scope grammar where intersection is not the greatest lower bound its callers assume.

The policy engine, adversarially #

plan/policy.md and docs/write-pipeline.md's whole design rests on "adding a policy can never widen what is permitted." crates/didbot-policy, crates/didbot-policy-regex, crates/didbot-policy-source and the gate plus per-repository queue in crates/didbot-pds landed in one day, from four agents working in parallel, and had never been attacked before this pass.

What held, tried adversarially and not merely read: shuffling a fixed outcome set through every rotation (didbot-policy's shuffling_outcomes_cannot_change_the_verdict_severity) — genuinely reaches combine, not just a hand-built list, since evaluate builds exactly that (PolicyId, Outcome) sequence from PolicyTree::decision_candidates; an unusable predicate (bad syntax, an unrecognized language, an unresolved blob) denies only its own declared applicability, never everything and never nothing; a stale poll observation (equal or lower revision) never overwrites a stored one, including out-of-order arrival; a failed poll tick leaves OperatorView untouched rather than clearing it; a declared blob size over the limit is refused before a single byte is fetched, and a streamed one that lies about its size is caught mid-stream; KindSet::of never matches SubjectKind::Unknown no matter which four kinds it names, so a typo'd or future account kind falls to the narrow default rather than escaping every rule; and the freeze-drain path never lets PolicyGate::observe see a denial the freeze itself caused, which is exactly what would let a metapolicy loop.

• An operator policy's collections and pathPrefixes are unchecked strings — no NSID or path-shape validation at all. didbot-policy-source::record::parse_scope_strings accepts any JSON string. Deny-only makes the failure mode benign in direction (a typo'd collection name matches nothing, which narrows enforcement rather than widens it — it can never let something through a correctly named policy would have caught), but it is a silent gap in exactly the way plan/policy.md's "The operator sees the tree" section anticipates: an operator who typos app.bsky.actor.proflie gets a policy that compiles cleanly, registers in the tree, and denies nothing, ever, with no signal anywhere that it is dead. Not fixed here — it is a dashboard/reporting gap (plan/policy.md's own unimplemented "operator sees the tree" section is the natural home for a "this policy has never matched a write" surface), not a security defect external input can trigger.

Provisioning: two writers racing one identity #

didbot-pds's provisioning sequence (Provisioner::provision) does a name reservation, a DID mint, a DNS write, a key generation, two record writes and a ledger append, several of them irreversible, and had never been attacked concurrently — only sequentially, the way crates/didbot-pds/tests/naming.rs and provisioning.rs already did before this pass. What held and what did not, from actually racing it with real threads rather than reasoning about it:

• Held: two requests minting the same agent_id. AccountStore::insert is a single check-and-insert under one mutex, so exactly one of two concurrent provision() calls for the same DID gets past it. The other's rollback withdraws the DNS record it believes it just published — the dangerous case would be that rollback deleting the winner's now-live DNS record — but InMemoryDns/LoopbackDns/WildcardDns all refuse a second publish for one hostname (DnsError::AlreadyPublished) rather than overwriting, so the loser's own dns.publish fails first, before it has touched the store, the ledger or anything worth rolling back. No test added: this is a property of Records::insert's existing publishing_twice_is_a_collision_not_an_overwrite test, already covering the mechanism this depends on.
• Held: naming through Naming::issue. NameRegistry::claim checks and claims a name under one mutex, so two concurrent provisions cannot both walk away believing they hold the same generated or hinted name.
• Fixed: two requests minting different agent_ids for the same caller-asserted handle, with no Naming configured. check_requested_handle scans the account store and the actual claim (AccountStore::insert) happens several steps later — a mint, a keypair, a DNS publish, a ledger open — far enough apart that two concurrent requests could both pass the scan before either was stored, minting two DIDs whose documents both claim one handle: exactly the situation two_accounts_may_not_claim_one_handle asserts is refused, reached through concurrency instead of sequencing. handle_did can then answer for only one of the two, permanently breaking the bidirectional handle/DID promise for whichever account it does not answer for, indistinguishably. Closed with Provisioner::handle_lock, serializing the check against the claim the way NameRegistry::claim already does for a deployment that names its own agents. Regression test: two_accounts_racing_for_one_handle_concurrently_still_only_one_wins in crates/didbot-pds/tests/naming.rs, real threads on a Barrier, matching the pattern records.rs's a_concurrent_write_between_the_check_and_the_commit_does_not_let_two_batches_win already uses for this class of race. Mutation-tested: reverting the lock (git stash the fix) fails the test 5/5 runs; restoring it passes 3/3.
• Held: the e-stop and lifecycle gates. docs/write-pipeline.md's ordering — e-stop before lifecycle before anything is parsed — is exactly what crates/didbot-serve/src/routes.rs's provision_agent does, and because both checks run before registry.provision() is ever called, "a refusal leaves nothing behind" is trivial rather than tested: nothing was started. Already covered end to end in crates/didbot-serve/src/tests.rs (Revoke, Pause, and Pause-thrown-by-a-lapsed-operator-claim each refuse provisionAgent with Halted, and Pause is shown leaving an already-issued token alone).

Left open, in rough order of how much a deployment should care:

• A rollback's DNS withdraw is best-effort and unaccounted for. Every rollback path in Provisioner::provision (a failed ledger open, a failed store insert, a failed registration-record publish) withdraws the DNS record(s) it published and logs, but does not return, a withdraw failure. The name is always released regardless — correct, since release_name never touches DNS — but a DNS provider that is unreachable during rollback (the same outage that plausibly caused the original failure) leaves a stray record pointing at a DID nothing will ever serve again, with no queue or sweep that revisits it. Not a security issue — the record is inert, not a takeover — but worth a reconciler pass; Route53Dns::resync, already flagged above as having no callers, is the natural place this would be caught if it were ever invoked.
• Naming::issue's hint path and its from-scratch path do not budget attempts the same way. A caller-hinted name that collides falls straight through to the generation loop with a full fresh budget of attempts; nothing bounds the combined cost of "try the hint, then try attempts more", which is fine at today's DEFAULT_ATTEMPTS = 16 but is a sharp edge for a deployment that raises it a lot expecting a linear cost per request.

The AWS surface, driven without an AWS account #

Every AWS interaction this deployment makes is this repository's own code: crates/didbot-dns/src/route53.rs signs its own SigV4 requests and reads IMDSv2 itself, and no AWS SDK is in the dependency graph. That puts the whole surface inside reach of a test, and didbot_dns::aws_fake — behind the aws-fake feature, a dev-dependency in didbot-pds and didbot-tls — is what it is driven against: a hosted zone holding record sets, applying change batches atomically, paginating a listing, taking a change through PENDING before INSYNC, and refusing a CREATE collision, a DELETE value mismatch, throttling and the record-set quota the way the service does, plus an instance metadata service issuing credentials that expire. The ACME half of a deployment keeps its own fake, FakeAcmeServer in didbot-tls's tests/support; tests/dns01_zone.rs uses both at once.

The suites, and the property each one holds:

• crates/didbot-dns/tests/route53_zone.rs — the round trip. A record the provider writes is one a second provider reads back over the listing wire format; a PENDING change is not listed until it is INSYNC; a throttled call is retried into one record set rather than several; a refusal is classified into the [DnsError] a caller can act on; a 25-record zone at 7 per page is read in four calls; a request signed with a lapsed credential is refused until instance metadata is read again.
• crates/didbot-pds/tests/route53_seam.rs — the store, the zone and the ledger, asserted together after each of: a slow zone, a throttled one, one that refuses the write outright, a store that fails once the zone has already been written, a credential that lapses mid-run, the record-set quota reached, and a run that stops between the DNS write and the store insert — that last one reconciled by didbot_reconcile's survey, which names the record the stopped run left.
• crates/didbot-tls/tests/dns01_zone.rs — a combined apex-and-wildcard order whose two DNS-01 values are live together in the hosted zone while the CA validates and gone from it after, and a zone that refuses the challenge write, whose refusal names the record and whose third failure is what RenewalTracker reports as a warning.

Every test in the three carries the mutation that makes it fail, in its own doc comment, and each was run.

Done #

• A second spelling of a denied client. DenyClient compares client_id as text, so https://Example.com/app would pass a rule written for https://example.com/app. didbot_policy::ClientId holds only the canonical spelling, POST /oauth/par refuses every other one as invalid_client, and every subject carries a ClientId. Tests: client_id::tests in crates/didbot-policy and a_second_spelling_of_a_client_id_is_refused_at_sign_in in crates/didbot-serve/tests/denied_app.rs.

• The AWS surface, driven without an AWS account. See the section of that name above for the fake, the three suites and what each holds.

• Two concurrent provisions racing for one caller-asserted handle. See "Provisioning: two writers racing one identity" above for what was attacked, what held, and what handle_lock closes.

• A caller-asserted handle more than one label below the zone. check_requested_handle accepted any handle hostname_is_at_or_below the zone, which allows any depth — deep.quernstone.agents.example passed as readily as quernstone.agents.example — but a deployment answers the zone with a single-label wildcard DNS record, which matches exactly one label per RFC 1034. A multi-label handle would mint an account, publish a document claiming the handle, and pass the internal document.claims_handle / handle_did agreement check, while no real request for that handle could ever route to this server: unreachable, not merely unpublished. Confirmed by provisioning one and checking the well-known lookup never answers before fixing it. check_requested_handle now refuses a handle with more than one label past the zone the same way it already refuses one outside the zone. WildcardDns::accepts in crates/didbot-dns mirrors the same overly permissive shape (ends_with, not "exactly one more label") and is left open — two other sessions are editing crates/didbot-dns/ and crates/didbot-tls/ concurrently. Regression test: a_requested_handle_more_than_one_label_below_the_zone_is_refused in crates/didbot-pds/tests/naming.rs. Mutation-tested: reverting the check (git stash the fix) fails the test, restoring it passes.

• A request naming a sibling zone's hostname, driven through the actual server, from HTTP request to whichever component decides containment, confirming the same string that would pass a unit test's call to the containment function is refused at the point a real attacker would submit it — a DID document lookup, a handle resolution, or a vouch naming a hostname just outside the zone. Also covers a hostname containing the zone as a substring rather than a suffix, an uppercase sibling (proving validate_handle refuses it outright rather than a silent lowercase fold), and the overlapping-zone case reached through ZoneManager::add_zone rather than ZoneRegistry::new directly. See crates/didbot/tests/zone_containment.rs.

• A proof minted for one key, replayed with a different key's signature but the original jti. The adversarial case is the combination — replay and a key swap in the same attempt — which is what an attacker holding a captured proof but not the private key would actually try. Refused. See crates/didbot-serve/src/tests.rs's dpop_binding module.

• A token minted with cnf.jkt bound to key A, presented with a valid, well-formed proof signed by key B. Found first as something sharper: OAuthTokenStore::validate_access took no proof parameter at all, so a bare stolen token authenticated with zero proof of possession. Its signature now requires a verified thumbprint, so the unsafe call does not compile, and the token's cnf.jkt is checked against it on the repo write path.

• The nonce and replay window survive a request that crosses authorize.rs and token.rs. Reachable now that OAuthState wires the real verifier: dpop_seam.rs's note that the replay window and the clock-skew window are the same number holds across a proof that has travelled both stages, rather than one fed to a single verifier call.

• An authentication bypass on an empty operator secret. The comparison was a bare ct_eq, and ct_eq of two empty slices is true, so a deployment whose secret file came back empty authenticated Authorization: Operator — a header anyone can send — as the operator. A failed SSM fetch redirected at the file's path left exactly that zero-byte file. Closed by removing the operator shared secret: crates/didbot-serve/src/auth.rs holds no operator credential for anything to compare. The test that missed it asserted two specific secrets and None rather than a property over a corpus.

• Shell operator precedence in provisioning. dnf install -y docker || apt-get update && apt-get install -y docker.io groups as (dnf || apt-get update) && apt-get install, so on a host where dnf succeeded the apt install ran anyway, failed, and aborted cloud-init under set -e before the service was ever installed. Braced.

• A live e-stop socket could be unlinked out from under its listener. remove_if_stale treated any connect(2) failure as "nothing is listening" — EACCES, a full backlog, EMFILE — and removed the file, after which a second process binds a fresh one and silently takes over e-stop administration while the first still believes it owns the path. Only ECONNREFUSED means stale now, and the path must actually be a socket. The function's own doc already said it should refuse to guess; the code did the opposite.

• The scope ceiling, as an invariant rather than a table. Eight tests in crates/didbot-serve/src/oauth/scope.rs's ceiling_boundary module, over every pairing of a corpus built out of adjacent atoms: nothing granted is outside the ceiling, nothing granted is outside the request, re-applying a ceiling changes nothing, reversing a ceiling grants the same thing, an empty ceiling admits nothing, disjoint atoms are refused and overlapping ones narrow to an exactly-stated grant. Each was checked by mutating the production rule it covers and confirming a failure; two mutations survived the first corpus (intersect_size taking the larger cap, intersect_aud dropping a bound) because every corpus pair was decided by Scope::intersect's two contains short-circuits and never reached the arms below them, which is its own lesson about what a corpus has to contain.

• Zone containment adversarial suite added (crates/didbot/tests/zone_containment.rs); see that section above for what it covers.

• DPoP binding adversarial suite added (crates/didbot-serve/src/tests.rs's dpop_binding module). It found three things — the verifier was never wired, the server's htu could not match a conformant client's, and validate_access took no proof — and all three are closed; see that section above.

• TreePolicyGate never told any evaluator about a freeze, an unfreeze, or a deletion. PolicyGate::froze/unfroze/deleted default to no-ops, and TreePolicyGate — the only implementation backed by a real PolicyTree — never overrode any of the three, so a write-tripped freeze, an operator's unfreeze, or an account deletion never reached a single Evaluator::lifecycle call. plan/policy.md names the failure mode directly: "An evaluator can never unfreeze. Only an operator, explicitly. But the evaluator must be told, or the state that triggered the freeze survives the unfreeze and slams it shut again on the next attempt" — and the same paragraph for deletion, state that "survives" is per-account memory nothing releases. Invisible today only because the one evaluator that exists (didbot-policy-regex) is stateless and its lifecycle is already a no-op; the first stateful evaluator (a rate limit, or the mention/opt-out metapolicy plan/policy.md motivates the whole design with) would have been unfreezable in practice. Fixed: PolicyTree::evaluators() exposes the tree's full evaluator set (lifecycle events are about an account, not a subject shape, so nothing to route through the applicability index), and TreePolicyGate::froze/unfroze/deleted now call every evaluator's lifecycle. Test: policy_tree::tests::froze_unfroze_and_deleted_reach_every_evaluator_in_the_tree in crates/didbot-pds/src/policy_tree.rs — fails against the trait defaults (confirmed by reverting the three method bodies to let _ = ... and rerunning), passes with the forwarding in place.

• The blob sweeper cannot tell a fresh data directory from one whose log is missing. DurableStore::open_with_limits in crates/didbot-pds/src/durable.rs refuses with WalError::Orphaned when the directory holds blob bytes and no log or checkpoint that names anything, before and after replay, and reconciles blobs only when there was state to read. Test: blobs_without_a_log_refuse_to_open_rather_than_sweeping_themselves_away in crates/didbot-pds/tests/durability.rs.

• OwnershipPoll read any HTTP 400 as "no operator claim". fetch_claim_from in crates/didbot-serve/src/ownership_poll.rs reads a 400 as absence only when the body names RecordNotFound; any other 400 is an unreachable repository. Test: a_400_that_is_not_record_not_found_is_unreachable_not_an_absent_claim in the same file.

• A freeze names who froze it. A freeze is a Lock tag on the account row (AgentAccount::locks, crates/didbot-pds/src/lockout.rs), and each tag carries the Party that hung it: operator, policy, the account itself, or a parent. The tag outlives the write that tripped it. A policy outcome hangs Lock::Quarantined under Party::Policy and records a LedgerEvent::Locked carrying the evaluation id (crates/didbot-pds/src/provision.rs, the gate's froze).