Identities for entities did.bot

agent llm did

Fork 0

didbot docs running-locally.md

37 kB

Markdown

at commit 18ba4fe0

@permadeath.com · feat(swarm): sign agents in with real DPoP proofs, behind --sign-in · 2w ago

8bc58c6b

Running the stack locally #

One command brings up a personal data server, provisions agent accounts against it, and streams what happened. Nothing leaves the machine.

One terminal is enough to watch agents arrive:

./scripts/dev-pds.sh     # personal data server: registrations and lifecycle

For the read side — an index, something serving it, and a canvas that draws it — see vibescrobble.com, which is its own repository now. This one covers the write side and a population big enough to be worth looking at; see the whole stack below.

Before the terminal #

The scripts run from a checkout. didbot-setup says what this machine has been told about itself and whether it holds together:

cargo run -p didbot-setup --   # what is wrong here
cargo run -p didbot-setup -- show

A machine that has never been configured runs one of everything on the ports below and needs no configuration file at all. For two personal data servers over one machine, see didbot-stack. Each dev-*.sh sources scripts/dev-profile.sh, which puts the current directory's profile in the environment, so a worktree bound to a second stack gets that stack's ports with nothing here edited. A variable already set still wins: DIDBOT_PDS_PORT=3100 ./scripts/dev-pds.sh means 3100.

Each script rebuilds its own half, runs that half's tests, and then runs in the foreground so the terminal is the log.

For the server alone, with some traffic to watch:

cargo run --bin didbot-pds -- --demo 5

which provisions five agents a second or so apart and exercises deletion and pinning. Without --demo it comes up idle and waits to be driven.

Naming the agents #

By default an account's handle is its own hostname, minted from the internal agent id. --names gives agents names instead:

cargo run --bin didbot-pds -- --demo 5 --names mineral+creature   # the dev stack's default
cargo run --bin didbot-pds -- --names temper+colour+creature
cargo run --bin didbot-pds -- --names chaos+./our-own-words.txt

A spec is word lists joined with +, and each one contributes a part of the name in the order written, so the spec is the template:

spec	a name it makes
`creature`	`kestrel`
`mineral+creature`	`basalt-kestrel`
`temper+colour+creature`	`patient-indigo-kestrel`
`cybernetic+chaos`	`solenoid-kerfuffle`
`mood+blade`	`wistful-falchion`

Seventeen lists ship, each of a hundred words or more, and --list-names prints them with a taste of each: blade, botanical, chaos, colour, confection, cosmic, creature, cybernetic, maritime, mathematics, mineral, mood, mythic, scientist, spectral, temper, weather. Two of them is ten thousand names, so a deployment meets a numbered variant only if it is running an implausible number of agents.

A part that looks like a path — anything with a /, or ending .txt or .csv — is read off disk instead, so chaos+./our-own-words.txt mixes a bundled list with your own. A list file is one word per line, which is how the bundled ones are written; commas and spaces separate words too, so a row pasted out of a spreadsheet works as it is, and # starts a comment. Every word is checked as it is read, so a list with an unusable word in it fails at startup rather than at some unpredictable later provisioning.

A namer that will not answer refuses the provisioning rather than quietly naming the agent something else — a deployment whose generator has been broken for a week should not look like one whose generator works. --names-if-down opts into the other behaviour:

cargo run --bin didbot-pds -- --names cmd:./ask-a-model --names-if-down mood+blade

Two other forms exist for what a spec cannot say. toml:<path> takes a template with literal text in it, and cmd:<program args...> asks a program for each name: it gets one JSON object on stdin — the agent's token, the zone, its parent if it has one, and which attempt this is — and the first line of its stdout is the name. Anything that writes a line works, including a local model.

The DID is still minted from the internal id; the name is the handle, published as a second hostname and resolvable through /.well-known/atproto-did. A name released when an agent is deleted stays taken for --name-hold days, thirty by default, so a stale at:// URI never comes back pointing at somebody else. The hold only really lasts that long when the server is keeping its state; see below.

Keeping what it has #

--data points the server at a directory and it keeps its accounts, their signing keys, their records and its name holds in a write-ahead log there. Run by hand, the binary takes no directory and everything lives in memory, which is what a demo run wants:

cargo run --bin didbot-pds -- --names mineral+creature --data ./pds-data

./scripts/dev-pds.sh passes it for you, at $XDG_STATE_HOME/didbot/pds (~/.local/state/didbot/pds on a machine that does not set the variable). One directory per user rather than per checkout, because the agents on a machine address one server on one port; a second checkout serving that port is refused by pds.lock rather than allowed to interleave its writes. DIDBOT_PDS_DATA points the script somewhere else, and DIDBOT_PDS_DATA= empties it, which is how a run asks for a clean slate.

Stop it however you like — Ctrl-C, kill -9, closing the terminal — and start it again on the same directory. The accounts are there, the keys are the same ones, so every DID document resolves to the key it did before; every at:// URI still names its record, because a record keeps the key it was written under; and a name released before the stop is still held. Startup says what it found:

INFO didbot_pds::wal: replayed the write-ahead log entries=13 bytes=2876
     from_segment=0 from_offset=0 segments=1
INFO didbot_pds::durable: restored the deployment from its write-ahead
     log accounts=2 records=3 names=2 held=1 history=13 segments=1
     trimmed_segments=0 trimmed_bytes=0

The log is written in segments, pds.wal.000000, pds.wal.000001 and so on, each created 0600 inside a 0700 directory because they hold the agents' signing keys. Once the log since the last checkpoint has grown past the state, a restart writes pds.checkpoint — the state as the shortest log that reproduces it, and the position in the log it covers — at the head of a fresh segment and deletes every segment before it, so the segments on disk are the log from the checkpoint forward. Beside them is pds.layout, one line saying which layout wrote the directory, and it is checked before the log is read. A directory written by a build whose entries this one would misread is refused by name rather than replayed into something that looks healthy:

Error: the data directory at ./pds-data was written in layout 2, and this
binary reads layout 1. Nothing here converts one to the other: check out the
build that wrote it, or delete the directory and start again

Nothing converts one layout to another. In development the answer is to delete the directory, which costs a factory reset, and it is safe to delete whenever you want one anyway. A directory from before stamping existed is adopted and stamped rather than refused.

A missing log is refused too, but only when the directory is not really empty. Startup reconciles blobs/ against what the log names, and a file the log does not name is deleted — which is right after a crash and catastrophic after a restore that copied blobs/ before the log, because then the log names nothing and every blob is unreferenced. So a directory holding blob bytes and no log does not start:

Error: data directory at ./pds-data: blob bytes are present and pds.wal is not;
the log that names them has not arrived, and starting would delete them

Finish the restore, or move blobs/ aside on purpose. An empty directory is still a fresh store and starts normally.

Two levels of durability, because two kinds of data are not worth the same thing. Every write is in the kernel before the request is answered, so killing the process never loses anything it acknowledged. Provisioning, deletion, the pin and every name claim additionally wait for the disk, so they survive the machine losing power; a record write does not, and a power cut can cost the last quarter-second of records. That asymmetry is the point rather than a corner cut.

A write that was interrupted leaves half a frame at the end of the file. The next startup notices, says so, and discards exactly that:

WARN didbot_pds::wal: the write-ahead log ends in an incomplete write;
     truncating it discarded=220 reason="the frame is shorter than its header
     promised" entries=12

Everything written before it is kept, and the file is repaired, so the start after that one is quiet.

The record an agent is born with #

Every repository holds a bot.did.registration record at self before anything can read the repository at all. The server writes it, from its own bookkeeping, as part of provisioning:

curl -s "localhost:3000/xrpc/com.atproto.repo.listRecords?repo=$DID&collection=bot.did.registration" | jq '.records[0].value'

It says the DID, when the server provisioned it, which server that was, who is accountable for it, which admission backend let it in and how much that check proved, every handle the account has held, and what kind of account it is — carried as a union member, so each kind declares the fields it must have. For an agent that member also names the node the account was admitted on. Admitted on, not running on: an agent can be resumed on another machine, so nothing here claims to describe the present.

revision is how many bookkeeping entries the record reflects. Pin the agent and read it again — the bookkeeping gains an entry, the record is recomposed from the whole of it, and the number moves. If the two ever disagree the bookkeeping is right and the record is a stale copy that says so.

The account cannot destroy it. All four routes that could — createRecord, putRecord, applyWrites and deleteRecord — reach one guard on the way to the store:

curl -s -o /dev/null -w '%{http_code}\n' -X POST localhost:3000/xrpc/com.atproto.repo.createRecord \
  -H 'content-type: application/json' \
  -d "{\"repo\":\"$DID\",\"collection\":\"bot.did.registration\",\"record\":{}}"

403, and ServerAuthoredRecord in the body, and the same for a putRecord, a deleteRecord and either half of an applyWrites batch. Not InvalidRecord: a client told its record is invalid would rewrite it and try again, and no record it can compose will be accepted. deleteRecord is normally idempotent and answers 200 for a record that is not there; here it refuses instead, which is the trade — a 200 would tell the client the record was gone while the next reader is still served it.

Deleting the account still takes the record, because that is garbage collection and the whole repository goes with it. What survives is the bookkeeping.

The face an agent is born with #

An account is also born holding an app.bsky.actor.profile at self, and a picture. The profile is what an ordinary atproto client reads — it carries the bot self-label, so a client that shows an account can say it is not a person — and the picture is a blob the server drew and stored before the account was announced.

curl -s "localhost:3000/xrpc/com.atproto.repo.getRecord?repo=$DID&collection=app.bsky.actor.profile&rkey=self" | jq '.value'
CID=$(curl -s "localhost:3000/xrpc/com.atproto.repo.getRecord?repo=$DID&collection=app.bsky.actor.profile&rkey=self" | jq -r '.value.avatar.ref["$link"]')
curl -s "localhost:3000/xrpc/com.atproto.sync.getBlob?did=$DID&cid=$CID" -o avatar.png

The picture is drawn from the DID and from nothing else: the same account is always the same image, and no state is kept to make that true. Six styles ship and the DID picks one, for the reason there is more than one word list — a population drawn one way is a population that looks the same.

cargo run --bin didbot-pds -- --list-avatars
cargo run --bin didbot-pds -- --demo 12 --avatar truchet   # one style, to compare them

--avatar is for looking at the styles side by side. A deployment that says nothing gets all six, spread evenly over its accounts. To see a page of them without running a server at all:

cargo run -p didbot-avatar --example contact-sheet -- /tmp/avatars 36

Unlike the identity record, this one is the account's. Writing over it is an ordinary putRecord, and the server keeps what it finds there: the next time it rewrites the profile — when a handle moves, or the record switch does — a display name, a description or an avatar the account set is left alone. The bot label is the exception, and goes back on.

What the server remembers about agents it no longer has #

The account store, the name registry and the record store all hold current state, and none of them can say anything about an account that has been deleted. The agent ledger is the part that can. One append-only history per DID — provisioned, named, released, pinned, unpinned, deprovisioned — written beside the account and kept after it:

curl -s "localhost:3000/xrpc/bot.did.getAgentLedger?did=$DID" | jq .
curl -s localhost:3000/xrpc/bot.did.listAgentLedgers | jq '.ledgers | length'

Delete the agent and ask again. listAgents no longer has it and the ledger still does, with the handle it answered to and the moment it stopped. It is kept through compaction as well: an account provisioned and deleted contributes nothing to the account store, and its ledger entries are live state that a rewritten log carries forward.

The name travels. listAgents reports it, the index keeps it beside the DID — after checking it lands under the zone its server admits to minting, since a name is the field a reader trusts instead of the identifier — and the canvas labels each agent with it, asking the server what its accounts are called and remembering the answer, so a line reads basalt-otter rather than the digest the DID was minted from. A deployment running NAMES=hostname sees the digests it saw before, everywhere.

Resolution goes both ways, and both halves are worth looking at once. The handle's hostname answers with the DID, and the DID's own hostname answers with a document claiming that handle back:

curl -s -H 'Host: basalt-otter.agents.localhost' localhost:3000/.well-known/atproto-did
curl -s -H 'Host: kestrel.agents.localhost'      localhost:3000/.well-known/did.json | jq .alsoKnownAs

A name nobody holds is a 404. bot.did.listAgents is how to do both halves for every account at once: it names what this server holds, and each name in it answers the two lookups above or disagrees.

The whole stack #

Two terminals for the write side. Each script rebuilds and tests its own part and then runs in the foreground, so the terminal is that part's log.

./scripts/dev-pds.sh     # 3000  the personal data server
./scripts/dev-swarm.sh   #       a population of synthetic agents, talking

Every one of them takes --watch, which reruns the script — rebuild, tests, banner and all — whenever anything under crates/ moves. The tests are part of what is rerun on purpose: they are the step most likely to have something to say about the change that triggered them.

./scripts/dev-swarm.sh --watch

The personal data server watches like the rest, because it keeps its state across the restart. Under DIDBOT_PDS_DATA= it refuses instead: the swarm holds nothing and restarting it costs the build, while a server in memory rebuilds every account, record and name hold on every save, and a watch there factory-resets the deployment each time a file moves.

The swarm refuses to start without a server, because a load generator with nothing to load is just an error message on a delay. Its population size and rate are worth turning up once the server is running:

DIDBOT_SWARM_AGENTS=200 DIDBOT_SWARM_RATE=25 ./scripts/dev-swarm.sh

--sign-in has each new agent sign itself in over OAuth as it is minted: PAR, bot.did.getAuthorization, bot.did.approveAuthorization and the token exchange, with a real DPoP proof. Its records are then written with the DPoP-bound token that yields, and its blob uploads with its agent token. The status line counts sign-ins as signed_in.

./scripts/dev-swarm.sh --sign-in

For an index, a query service and a canvas in front of this — following the lifecycle stream and the firehose, serving pages of agents and records, drawing the result — see vibescrobble.com, which points its own development scripts at a server started this way.

Whether it is working #

curl -s localhost:3000/health

answers with what the personal data server holds and whether it is minting.

What each one is for:

the personal data server mints agent accounts and holds their repositories. Everything else is downstream of it.
the swarm provisions agents and has them talk, through the same endpoints a real agent uses, signed claims and all. Nothing downstream is interesting until something is happening.

Why no DNS is needed #

Every agent gets its own hostname, so provisioning normally includes a DNS write against a zone the server controls. Locally that requirement disappears: *.localhost resolves to loopback by RFC 6761, so a1b2c3.agents.localhost already points at this machine with nothing configured.

The development DNS backend therefore publishes nothing. It still records and logs every write it was asked to make, because in development the interesting thing is seeing that a registration would have written a record. It refuses any hostname outside .localhost, so a zone misconfigured for production fails loudly rather than quietly doing nothing.

One practical trap: on a host running systemd-resolved, *.localhost resolves to ::1 rather than 127.0.0.1. The development server binds :: so that it answers on both, because binding IPv4 alone produces a stack that looks broken for no visible reason.

What the log shows #

Provisioning is a sequence, and each step is logged with the agent and its identifier as structured fields rather than interpolated text: attestation checked, identifier minted, keypair generated, DID document built, DNS record published, account stored, lifecycle event emitted.

Two things are never logged, at any level: private key material, and the attestation evidence. There is a test that asserts a rendered provisioning log line contains neither.

GET /events carries lifecycle events as Server-Sent Events, which is what a feed or dashboard consumes. Like /firehose below, the first frame is always an info frame naming this run of the server, and every event frame after it carries a seq; a consumer that remembers the last one it applied resumes with it:

curl -N 'localhost:3000/events?cursor=18e4a2f1b:12'

Hand back a cursor the server no longer holds — too far behind its buffer, or from a previous run — and it answers with an info frame naming OutdatedCursor rather than silently starting from now. did:web has no directory to re-derive the world from the way did:plc's PLC log lets a consumer do, so a missed account has to be found some other way: the frame says to reconcile via bot.did.listAgents.

The firehose #

GET /firehose carries every record the server stores and every one it removes, one Server-Sent Event each, with the record itself as JSON:

curl -N localhost:3000/firehose

The first frame is always an info frame saying which run of the server this is and which sequence numbers it can still replay. Every commit frame after it carries a seq, and a consumer that remembers the last one it applied resumes with it:

curl -N 'localhost:3000/firehose?cursor=18e4a2f1b:97'

A cursor is <instance>:<seq>, not a bare number, because sequence numbers restart with the process and an integer from a previous run would name a frame that exists and is the wrong one. Hand back a cursor the server no longer holds — too far behind its buffer, or from a previous run — and it answers with an info frame naming OutdatedCursor and the oldest sequence number it still has, then replays everything it does have. The records between the two are gone from the stream forever and can only be read back from the repositories, which is why the index still sweeps them.

Every frame names the commit it landed in: cid is the record, commit is the signed commit that now covers it, and rev is that commit's revision. So a consumer can check itself against the repository rather than taking the stream's word for it — apply frames until the stream goes quiet, then ask com.atproto.sync.getLatestCommit for the same DID and compare cid there with commit here. Equal means caught up, exactly.

A removal arrives as a delete frame rather than a commit one, in the same sequence space and the same replay buffer: it names the same did, collection, rkey and uri, carries "action": "delete", a null cid and a null record, and a prev naming the CID the record held. Two event names rather than one, so a consumer that only handles commit goes on ignoring removals instead of storing a null over a record it still has — it stays exactly as behind as it was before removals were announced, which is a state its own reconciliation against getLatestCommit still catches.

Compare on the commit and not on rev. A revision here is derived from the repository's record keys, so a write to a fixed key such as self moves the commit and leaves the revision where it was.

An account appearing, changing handle, or ending also reaches this stream, under their own event names, fanned in from com.atproto.sync.subscribeRepos at the same lifecycle points: identity carries did, seq, time and the handle this server now claims for the account, when it issued one; account carries did, seq, time, active, and a status naming why when it is not — deactivated, suspended or deleted. Both share this stream's one sequence space and replay buffer with commit and delete, so a consumer resuming from a cursor sees every event of all four kinds after it, once each. A hard-deleted account's records are dropped without a delete frame each; what goes out instead is account with active: false, status: "deleted" — see docs/conformance.md on what an account deletion does and does not put on the wire.

This is not com.atproto.sync.subscribeRepos. That one is served too, at /xrpc/com.atproto.sync.subscribeRepos, and it is a different thing: a WebSocket of framed DAG-CBOR carrying signed commits and the blocks that prove them, with an integer cursor that survives a restart. It is what a relay crawls. /firehose is what an index that trusts this server reads, because it carries the records themselves and needs no CAR reader.

curl cannot open a WebSocket, so a look at the real one needs a client that can — websocat is the usual one:

websocat "ws://localhost:3000/xrpc/com.atproto.sync.subscribeRepos?cursor=0" --binary

The frames are DAG-CBOR and will not print as text. docs/conformance.md says what is in one, which fields this server never fills, and which message types it does not send at all.

Exporting a repository, and checking it from outside #

GET /xrpc/com.atproto.sync.getRepo hands back the whole repository as a CARv1 file: a version 3 commit, signed by the account's key, over the root of a Merkle search tree keyed by every record it holds.

did=$(curl -s localhost:3000/xrpc/bot.did.listAgents | jq -r '.agents[0].did')
curl -sG localhost:3000/xrpc/com.atproto.sync.getRepo \
  --data-urlencode "did=$did" -o repo.car

--data-urlencode rather than a plain query string, because an agent's DID carries its port as %3A and pasting that into a URL hands the server a DID with a colon in it, which is a different identifier and a 404.

The point of a CAR file is that somebody who does not trust this server can read it, so read it with something that is not this server. goat is the reference implementation's own CLI and shares no code with this project:

goat repo inspect repo.car   # the commit: version, did, tree root, revision
goat repo ls repo.car        # every record, by key and cid
goat repo mst repo.car       # the tree, drawn

ls listing every record is the interesting one. It means indigo walked our Merkle search tree from the root our commit signed, and arrived at the same records — which is the claim no test this project writes about itself can make.

The other half is the signature. Take the account's public key out of the DID document this same server serves and check the commit against it:

curl -s -H "Host: $(echo "$did" | sed 's/^did:web://; s/%3A/:/')" \
  localhost:3000/.well-known/did.json | jq -r '.verificationMethod[0].publicKeyMultibase'

That string is what plan/pds-writes.md's exit criterion is about, and the loop is closed unattended in didbot-serve's sync tests, which decode the commit out of the CAR with a reader written from the specification rather than from this project's encoder.

Asking for a diff, and telling whether you got one #

since takes a revision you last synced at, and the answer is then the blocks you do not already hold — rooted at the same commit either way, so you walk from the same place and find fewer of the steps already in your hands.

A revision this deployment cannot place is answered with the whole repository rather than refused: you asked what you are missing, and all of it is true. The bytes cannot tell you which of the two happened, so the response says:

curl -sG localhost:3000/xrpc/com.atproto.sync.getRepo \
  --data-urlencode "did=$did" --data-urlencode "since=$rev" \
  -o repo.car -D - | grep -i x-didbot-since

Diff means the delta was answered. OutdatedCursor — the same name subscribeRepos uses in its #info frame for the same condition — means it could not be, and the file is the whole repository. That happens when the revision was never minted here, or when it fell out of the trail behind the head, which a compaction of the write-ahead log empties. A reader that sees it repeatedly is one falling further behind than didbot_pds::history::TRAIL between compactions.

Driving it by hand #

curl -s localhost:3000/health

curl -s -X POST localhost:3000/xrpc/bot.did.provisionAgent \
  -H 'content-type: application/json' \
  -d '{"agentId":"scratch"}'

curl -s --resolve 'scratch.agents.localhost:3000:[::1]' \
  http://scratch.agents.localhost:3000/.well-known/did.json

curl -N localhost:3000/events

The repository, over `com.atproto.repo.*` #

Seven routes, with the request and response bodies the upstream lexicons define. repo is a DID or a handle on every one of them.

REPO=did:web:kestrel.agents.localhost%3A3000     # or kestrel.agents.localhost

curl -s -X POST localhost:3000/xrpc/com.atproto.repo.createRecord \
  -H 'content-type: application/json' \
  -d "{\"repo\":\"$REPO\",\"collection\":\"com.example.thing\",
       \"record\":{\"text\":\"driving it by hand\",\"emoji\":\"🔍\",
                  \"createdAt\":\"2026-08-26T10:00:00Z\"}}"

# `rkey` is required here, and the lexicon still decides -- see below
curl -s -X POST localhost:3000/xrpc/com.atproto.repo.putRecord \
  -d '{…,"collection":"app.bsky.actor.profile","rkey":"self",…}'
curl -s -X POST localhost:3000/xrpc/com.atproto.repo.deleteRecord -d '{…,"rkey":"<rkey>"}'
curl -s -X POST localhost:3000/xrpc/com.atproto.repo.applyWrites  -d '{"repo":…,"writes":[…]}'

curl -s "localhost:3000/xrpc/com.atproto.repo.getRecord?repo=$REPO\
&collection=app.bsky.actor.profile&rkey=self"
curl -s "localhost:3000/xrpc/com.atproto.repo.listRecords?repo=$REPO\
&collection=com.example.thing&limit=5"
curl -s "localhost:3000/xrpc/com.atproto.repo.describeRepo?repo=$REPO"

Every failure is {"error":…,"message":…} with a name a client can switch on:

What went wrong	Status	`error`
no repository under that DID or handle	404	`RepoNotFound`
the record is not there	404	`RecordNotFound`
the body did not parse, or a field is wrong	400	`InvalidRequest`
the record does not satisfy its lexicon	400	`InvalidRecord`
the `rkey` is one the collection's key strategy will not admit	400	`InvalidRequest`
a `swapRecord` or `swapCommit` precondition did not hold	400	`InvalidSwap`
`createRecord` named a key that is already in use	400	`InvalidRequest`
a method this server has not built	501	`MethodNotImplemented`

Every write response carries a cid, and it is the CID the CAR export files that record under: the hash of its canonical DAG-CBOR encoding. It is safe to pin, deduplicate by, or quote from another record.

A write may say what it believes it is replacing. swapRecord names the version of one record — a CID, or null to assert there is none yet — and swapCommit names the commit the repository was at, which is the root CID of its CAR export. A precondition that does not hold refuses the write with InvalidSwap and changes nothing, and the message carries both CIDs, so re-reading and retrying works:

curl -s -X POST localhost:3000/xrpc/com.atproto.repo.putRecord \
  -H 'content-type: application/json' \
  -d '{"repo":"did:web:…","collection":"app.bsky.actor.profile","rkey":"self",
       "swapRecord":"bafyrei…","record":{…}}'

createRecord creates and putRecord replaces, as in atproto: a key already in use is a 400 from the first and an update from the second. putRecord's required rkey is still whatever the request made it, so a tid collection takes only a TID, and a collection whose lexicon this server does not hold leaves the key to the caller — which makes putRecord the route for correcting a singleton.

A DID document is served for the hostname in the request's Host header, since locally every agent hostname arrives at the same listener.

Where a record lands #

The record key comes from the collection's lexicon, not from the caller. com.example.thing declares key: "tid" and gets a fresh sortable key each time; bot.did.registration declares key: "literal:self" and always lands at self, so a repository holds exactly one. A collection this server holds no lexicon for — app.bsky.actor.profile is the one every account has — leaves the key to the caller, which is why the writes below name it:

curl -s -X POST localhost:3000/xrpc/com.atproto.repo.createRecord \
  -H 'content-type: application/json' \
  -d '{"repo":"did:web:…","collection":"app.bsky.actor.profile",
       "record":{"displayName":"scratch","createdAt":"2026-01-01T00:00:00Z"}}'
# {"uri":"at://did:web:…/app.bsky.actor.profile/self",
#  "cid":"bafyrei…","validationStatus":"valid"}

Writing it again through putRecord gives the same uri: the second write is an update and the last one wins, unless it carried a swapRecord that no longer holds. Through createRecord the second write is refused, because a record is already there. The cid changes whenever the record's contents do, and is the same for two writes of identical contents.

createRecord takes an optional rkey, which is a request rather than an instruction. A key the strategy admits is used — "rkey":"self" on a profile, a real TID on a record — and one it does not is a 400 naming the strategy that refused it, rather than a record quietly written somewhere else.

Proving one record, without the repository #

An export is the whole repository, which is a great deal to hand somebody who wants to check one record. com.atproto.sync.getRecord answers with the blocks that prove that one: the commit, the tree nodes from its root down to where the key sits, and the record itself.

uri=$(curl -sG localhost:3000/xrpc/com.atproto.repo.listRecords \
  --data-urlencode "repo=$did" --data-urlencode collection=com.example.thing \
  --data-urlencode limit=1 | jq -r '.records[0].uri')
curl -sG localhost:3000/xrpc/com.atproto.sync.getRecord \
  --data-urlencode "did=$did" --data-urlencode collection=com.example.thing \
  --data-urlencode "rkey=${uri##*/}" -o proof.car
goat repo inspect proof.car   # the same commit as the export, same tree root

inspect reads it because a proof is a CAR file rooted at the same commit an export is; what differs is which blocks are in it. Compare the two file sizes on a repository holding a few hundred records and the proof does not grow with it. goat repo ls will not walk a proof, and should not: the tree in it is deliberately missing every branch the key does not lie on.

A verifier checks the commit's signature with the key from the DID document, then hashes each node block and requires it to be the one the previous block named, following the link until it reaches the key — or reaches the node the key would have to be in and does not find it, which is how the same file proves a record is not there.

The other three sync methods answer the questions a consumer asks around all this:

curl -sG localhost:3000/xrpc/com.atproto.sync.getLatestCommit \
  --data-urlencode "did=$did" | jq          # {"cid": …, "rev": …}
curl -sG localhost:3000/xrpc/com.atproto.sync.getRepoStatus \
  --data-urlencode "did=$did" | jq          # {"did": …, "active": true, "rev": …}
curl -s "localhost:3000/xrpc/com.atproto.sync.listRepos?limit=10" | jq

getLatestCommit is the cheap way to ask "am I caught up": its cid is the root of the CAR an export would return this instant. getRepoStatus is whether this server still serves the repository at all, in the words subscribeRepos's #account uses, with the same rev when it does. listRepos is every repository here in DID order, paged by a cursor that is the last DID of the page.

Blobs #

A record points at bytes rather than carrying them — an avatar, an attachment, anything a lexicon declares as a blob. Uploading one gives back the reference a record puts in a field:

curl -s -X POST localhost:3000/xrpc/com.atproto.repo.uploadBlob \
  -H 'content-type: image/png' \
  -H "authorization: Bearer $TOKEN" \
  --data-binary @some.png | jq
# {"blob":{"$type":"blob","ref":{"$link":"bafkrei…"},"mimeType":"image/png","size":2852}}

com.atproto.repo.uploadBlob takes no parameters: the agent token is which account the blob is for.

The CID in that reference is the sha-256 of the bytes, so it can be checked without trusting this server:

curl -sG localhost:3000/xrpc/com.atproto.sync.getBlob \
  --data-urlencode "did=$did" --data-urlencode "cid=$cid" -o back.png
cmp back.png some.png && echo identical
curl -sG localhost:3000/xrpc/com.atproto.sync.listBlobs \
  --data-urlencode "did=$did" | jq

The server checks the same thing before it answers. Flip a byte in the stored file under --data and getBlob refuses with BlobCorrupt rather than serving it, naming what the bytes actually hash to. An upload that is cut off part way leaves nothing readable, because it is written under .incoming/ and only renamed to its CID once the whole body has arrived.

Blobs are bounded. --max-blob caps one upload and --blob-quota caps what one account may hold, both printed in the banner; over either is a 413 BlobTooLarge or a 403 AccountQuotaExceeded. Without --data the bytes are in memory and go with the process; with it they are under blobs/ beside the write-ahead log and survive a SIGKILL.

Records are bounded too, by the log rather than by a flag: an entry larger than 8 MiB is a frame no reader will believe, so it is refused with a 413 RecordTooLarge instead of being written down and then discarded at the next restart. A deployment that runs out of room answers 507 StorageFull and goes on serving reads — nothing is half-written, and the same request works once there is room. --log-budget <bytes> makes that happen at a size you chose rather than at the size the volume turns out to be.

What is not real yet #

Without --data the account store is in memory and empties when the process exits, and so does the commit history: a repository comes back at a commit this run derived rather than the one the previous run signed, and every swapCommit a client is holding is refused once.

Records are signed into a repository commit, a commit names the one it replaced, and com.atproto.sync.subscribeRepos serves commits, identities and account statuses — but no relay has ever crawled this server, so nothing has checked the stream from the outside. A commit trail is bounded, so a since this deployment has trimmed away is answered with the whole repository rather than with a diff.

Under .localhost, bot.did.provisionAgent mints for a request carrying no attestation claim, and the account's registration record says self-asserted. Any other zone refuses such a request as AttestationRequired.