modding.place / atproto-crates
A fork of atproto-crates.
Star
Fork 0
Watch
atproto-crates tools conformance
1 folder · 6 files
lib
.gitignore
ci.sh
package.json
pnpm-lock.yaml
README.md
run.mjs
README.md

Reference-client conformance harness #

Boots a real atproto-pds and drives the real @atproto/oauth-client-node through PAR → authorize → token → resource request → refresh against it.

tools/conformance/ci.sh          # build, install, run — exits non-zero on failure

That is the whole command. It builds pds, installs the pinned client, and runs the checks. --runs=N changes how many sign-ins the soak performs (default 20). To run against a binary built somewhere else — another commit, say — use node run.mjs --pds-bin=/path/to/pds after pnpm install.

Why this exists #

Two OAuth bugs shipped from this workspace and survived every test in it:

  • d7da6c7 — the DPoP nonce challenge on /oauth/par and /oauth/token was answered with RFC 9449 §9's resource-server form (401 + WWW-Authenticate) instead of §8's authorization-server form (400 + {"error":"use_dpop_nonce"}). @atproto/oauth-client builds its authorization-server fetch with isAuthServer: true and then recognises only the §8 shape, so it never retried and sign-in failed at its first request.
  • aaa5fa9 — SignaturePolicy::AnyS was a no-op on secp256k1, because k256 enforces low-S inside its own verifier. @atproto/jwk signs ES256K and does not normalise s, so roughly half of a conforming client's DPoP proofs were refused as "invalid signature".

Neither is subtle. Both were invisible from Rust, because the only client that ever spoke to this server was the Rust one: our DPoP client accepts either status and consults WWW-Authenticate first, and our signer always emitted low-S for a verifier that demanded it. Client and server agreed with each other and every test passed. The defects were visible only from TypeScript.

So the harness contributes exactly one thing: a client we did not write.

What it checks #

  1. Nonce challenge shape (RFC 9449 §8). A proof carrying no nonce is sent to /oauth/par and to /oauth/token; each must answer 400 with {"error":"use_dpop_nonce"} and a DPoP-Nonce header, and the retry carrying that nonce must get past the nonce check. Both endpoints, because fixing /oauth/par alone moves the failure to client.callback().
  2. A high-S ES256K proof is a valid proof. Two proofs are minted from one key over one request: (r, s) and (r, n−s). Both are valid ECDSA signatures over the same message under the same key and RFC 7515 blesses neither, so both must be accepted. The low-S request is the control — if it fails, the harness is wrong about how to build a proof and the high-S result cannot be read.
  3. Twenty cold sign-ins with the reference client. Each run constructs a fresh NodeOAuthClient — an empty nonce cache — and completes PAR, authorize, the code exchange, an authenticated getSession, and a forced refresh, asserting the refresh returned a different access token.

Cold is not a detail. fetchDpop stores the nonce off a response it did not understand as a challenge, so a warmed client succeeds on its second attempt and a broken challenge reads as flakiness rather than as a bug. And twenty runs is not superstition: the PDS advertises ES256K, compareAlgos prefers it unconditionally, so every run mints a secp256k1 DPoP key and signs several proofs with it. One run has an even chance of never producing the high-S form. Twenty do not.

The resource-server half of the nonce protocol (§9: 401 + WWW-Authenticate) is covered by check 3 rather than by a probe of its own. The two nonce spaces have separate HMAC domain separators, so the nonce the client cached during PAR is refused at /xrpc/, and the challenge has to be understood a second time — in the other shape. A server that answered both spaces the same way fails one check or the other whichever way it chose.

Proof that it detects the bugs it exists to prevent #

A harness that cannot catch its two founding bugs is decoration. Both were re-introduced by building pds from the commit before each fix and pointing the harness at it with --pds-bin:

build nonce shape high-S 20 cold sign-ins
d7da6c7^ — neither fix FAIL (/oauth/par answered 401) FAIL (high-S refused, low-S accepted) FAIL 20/20
aaa5fa9^ (= d7da6c7) — nonce fixed, high-S not ok FAIL FAIL 19/20
HEAD ok ok ok 20/20

The middle row is the one to look at. Nineteen of twenty sign-ins failed and one passed — that single lucky run is what a one-shot harness would have reported as a green build.

To reproduce:

git clone . /tmp/ablation && git -C /tmp/ablation checkout aaa5fa9^
(cd /tmp/ablation && cargo build -p atproto-pds --bin pds --features clap)
node run.mjs --runs=20 --pds-bin=/tmp/ablation/target/debug/pds

Give the clone its own target/. Sharing one with CARGO_TARGET_DIR to save the dependency build does not work: two checkouts of the same workspace hash to the same -C metadata, so the clone's libatproto_identity.rlib overwrites yours and your next cargo build links the old one without recompiling it. The harness catches that — an atproto-pds carrying the nonce fix and an atproto-identity without the high-S one reports exactly the middle row above — but you will spend an hour deciding whether the bug is back.

How it is wired #

run.mjs ── lib/plc.mjs      a did:plc directory for one server, in memory
        ├─ lib/pds.mjs      spawns the real `pds` binary on a loopback port
        ├─ lib/gateway.mjs  the reverse proxy the PDS expects to sit behind
        ├─ lib/proofs.mjs   DPoP proofs built by hand, for checks 1 and 2
        └─ lib/flow.mjs     the reference client, for check 3

Nothing links against the crates. The harness talks to the pds binary over HTTP, so it can be pointed at a build of any commit.

Three pieces of scaffolding stand in for things a local server does not have, and each is worth understanding before trusting a result:

The gateway. issuer_url() in crates/atproto-pds/src/oauth/metadata.rs derives the OAuth issuer, the resource of the protected-resource document and the htu a pushed or token request is checked against from the service DID, always over https. The reference client refuses an https: URL whose host is loopback (httpsUriSchema in @atproto/oauth-types). A PDS is therefore only reachable by a conformant client from behind a TLS terminator, so the harness gives the server the identity pds.conformance.test and maps that origin onto the loopback socket exactly as a terminating proxy would: the URL's scheme and authority are replaced, X-Forwarded-Proto and X-Forwarded-Host are added (the server runs with PDS_TRUSTED_PROXY_HOPS=1, which is what makes a resource request's htu come out as the public origin), and nothing else is touched. Status, headers and body are handed back verbatim. That rule is not negotiable: the 401-vs-400 bug was diagnosed with a shim that rewrote a status code, and a gateway that edits a response is a gateway that can hide the next one.

The PLC directory. createAccount mints a DID by POSTing a signed genesis operation to the configured directory, and the client resolves that DID back to a document to check that the token it was handed came from the server the account actually lives on. lib/plc.mjs answers both, rendering the operation the PDS submitted as the document it describes. It does not verify signatures or order a log — it is a fixture, not an implementation.

The consent POST. client.authorize() returns a URL a human would open. The harness posts the credentials to /oauth/authorize — the JSON endpoint the consent form itself submits to — and carries code, state and iss back to client.callback(). That is the only step of the flow the harness performs rather than the client.

That shortcut has a cost, and it has already been paid once. Posting the JSON directly means the consent page itself is never exercised — not the form, not the inline script that lifts its submit into that JSON, not the redirect the script assembles afterwards. Every request the harness makes to /oauth/authorize is one the page would have made correctly. So a bug living in the page is invisible here in the same way the two founding bugs were invisible to the Rust suite, and for the same reason: the only client that ever drives the consent page is a browser, and no browser runs in this harness.

One did get out that way. The script called preventDefault, then .json() on every response, and attached nothing to the rejection — so any answer that was not a JSON body (an empty-bodied 408 from the request deadline, a 503 from load shedding, an HTML error page from a proxy in front of the origin) left the page doing nothing at all, for ever. It was reported to us as a PDS that hangs. The invariants are pinned in oauth::consent's unit tests now, which is a weaker thing than running them: those tests read the rendered document, they do not execute it. A harness that drove a real browser through the page would be the honest version, and does not exist yet.

The loopback client identity (http://localhost?redirect_uri=…&scope=atproto) is a spec-defined development client, so no client-metadata document has to be served. The redirect URI is never listened on; the harness intercepts the code before a browser would be redirected anywhere.

Adding it to CI #

.tangled/workflows/ci.yml needs nodejs and pnpm in dependencies.nixpkgs and one step:

  # The Rust suite cannot see a divergence between this server and a
  # TypeScript client, because it is all one stack agreeing with itself. This
  # runs the reference OAuth client against a real `pds`.
  - name: "oauth conformance"
    command: tools/conformance/ci.sh

It takes about ten seconds after the build: three checks, twenty sign-ins, several hundred requests. Everything is on loopback and the network is never reached — the PLC directory, the client metadata and the handle resolution are all local.