benchmarking #

Benchmark notes should separate behavior domains and only compare equivalent work. If a measurement does not exist for the same unit of work, keep it out of the comparison table instead of filling the cell with a nearby but different operation.

references #

• ZDS: bench/main.zig
• Tranquil: crates/tranquil-store/benches/metastore.rs
• Official PDS: packages/pds/src/actor-store/record/reader.ts

current matrix #

The active apples-to-apples matrix is intentionally small:

• apply one record commit
• current record CID lookup

Adjacent probes are tracked separately:

• record-index list
• full record materialization
• blob/block put and get
• repo export
• sync event delivery

ZDS, Tranquil, and the official PDS have matching single-caller numbers for apply-one-record and current-CID lookup. ZDS and Tranquil also have direct 10/100 caller concurrency curves. Official-PDS batch probes stay outside the main matrix until they are measured through the same unit of work.

Current local comparison. The Tranquil rows are from a fresh local metastore run on June 18, 2026. The ZDS write row was remeasured after completing the lazy-MST adoption in the repo write path.

operation	callers	zds	tranquil	note
apply one record commit	1	1598 ops/s, p95 0.65 ms	267 ops/s, p95 4.13 ms	ZDS no longer eagerly loads the whole repo CAR
apply one record commit	10	2313-2525 ops/s, p95 11.2-12.2 ms	503 ops/s, p95 26.1 ms	ZDS row is repeated-run range
current CID lookup	1	523k ops/s, p95 2 us	116k ops/s, p95 12.6 us	ZDS ahead
current CID lookup	10	319.7k ops/s, p95 8 us	294.6k ops/s, p95 54 us	both use record-index lookup

Tranquil's built-in 1000-caller metastore row currently trips handler backpressure in the benchmark harness. Keep it out of the table until the harness handles backpressure as a measured result.

ZDS write authoring uses zat.mst.Mst.loadLazy with a SQLite-backed repo_blocks reader. This avoids reading every stored repo block into a CAR before each write while preserving the signed commit, MST update, record index, and sync event behavior.

read path #

ZDS follows the official PDS read shape for getRecord and listRecords: read the current record index row, fetch matching repo_blocks bytes, decode DAG-CBOR, and render JSON for the XRPC response. Tranquil's metastore list_records is a different unit of work because it returns only rkeys and CIDs. Keep those rows separate.

Current ZDS read breakdown on this machine:

operation	callers	ops	result
current CID lookup	10	10000	222k ops/s, p95 225 us
current CID lookup	100	20000	211k ops/s, p95 887 us
index + block fetch	10	10000	128k ops/s, p95 11 us
index + block fetch	100	20000	121k ops/s, p95 15 us
full getRecord	10	10000	38.4k ops/s, p95 173 us
full getRecord	100	20000	38.1k ops/s, p95 11.7 ms
full listRecords, limit 50	10	10000	21.1k ops/s, p95 57 us
full listRecords, limit 50	100	20000	19.5k ops/s, p95 78 us

Adjacent Tranquil list comparison:

operation	callers	zds	tranquil	note
list records, limit 50	1	22.5k ops/s, p95 58 us	32.9k ops/s, p95 37 us	ZDS full materialization; Tranquil index-only
list records, limit 50	10	18.2k ops/s, p95 97 us	38.9k ops/s, p95 271 us	not same unit of work
list records, limit 50	100	21.5k ops/s, p95 60 us	38.5k ops/s, p95 2.75 ms	not same unit of work

The official PDS probe on the same machine reports:

operation	result
apply one record commit	448 ops/s, p99 4.0 ms
current CID lookup	3.1k ops/s, p99 511 us
full getRecord	3.0k ops/s, p99 525 us
full listRecords, limit 50	635 ops/s, p99 2.4 ms

The official PDS batched CID lookup probe reports:

batch shape	converted throughput	batch p99
10 callers x 100 lookups	3.3k lookups/s	317 ms
100 callers x 20 lookups	3.2k lookups/s	630 ms

For tiny records, DAG-CBOR decode and JSON rendering are below useful timer resolution. Larger records and index-only list probes are tracked as separate comparison gaps instead of being folded into the current table.

permissioned data #

Permissioned-data measurements should stay separate from the existing repo, blob, sync, and HTTP route matrices. The experimental com.atproto.space.* surface has its own storage model, actor-local space state, writer repo state, record oplogs, credentials, and ranged blob reads, so mixing it into the public repo tables would hide the actual access pattern.

The first benchmark slice should be small and diagnostic rather than exhaustive:

• seed one self-owned private space and one writer repo.
• measure createSpace, record write, record read, listRecords, full-state getRepo CAR construction, ranged getBlob, and writer repo oplog catch-up separately.
• report p50/p95/p99/max for the concurrent paths, but keep write, read, blob, and oplog rows separate.
• compare to Daniel's permissioned-data branch only after we can run that PDS locally with the same seeded work.

The question for this slice is simply whether any route has an obviously bad query shape, full-table scan, unnecessary blob materialization, or avoidable global lock hold. It does not need to become a giant protocol benchmark suite.

HTTP route path #

just bench http starts a local ReleaseFast ZDS server and measures a seeded route matrix with ApacheBench. These numbers include HTTP request parsing, routing, response serialization, auth/session work, httpz connection handling, and whatever endpoint work the route performs. Keep them separate from the storage microbenchmarks above.

The current harness covers public metadata, OAuth metadata, session/account reads, preference reads, repo reads, sync reads, blob reads, and repo CAR reads. It deliberately does not loop over stateful write endpoints such as createRecord, applyWrites, uploadBlob, or passkey registration because a static ApacheBench body would mostly measure duplicate/error paths after the first request.

just bench websocket-churn covers the WebSocket lifecycle separately. It repeatedly opens and closes cohorts of subscribeRepos clients at the server's live-connection limit, then checks that detached connection workers return to the baseline thread count. A warm second round also guards against renewed, unbounded RSS growth while allowing the allocator to retain its first-round high-water capacity.

Current local route baseline after adopting httpz, with 5000 requests at concurrency 50 for normal routes and 500 requests at concurrency 10 for the larger blob/CAR routes:

route	httpz
/	15.0k req/s, p95 10 ms
/favicon.svg	40.2k req/s, p95 1 ms
/og-image	11.6k req/s, p95 5 ms
/xrpc/_health	31.9k req/s, p95 1 ms
/.well-known/did.json	38.9k req/s, p95 1 ms
/.well-known/atproto-did	38.5k req/s, p95 1 ms
OAuth metadata/JWKS	35.6k-37.0k req/s, p95 2 ms
oauth/authorize JSON	4.4k req/s, p95 28 ms
describeServer	38.7k req/s, p95 1 ms
createSession	7.4k req/s, p95 20 ms
getSession	23.8k req/s, p95 6 ms
getServiceAuth	24.5k req/s, p95 5 ms
account/security reads	20.5k-34.9k req/s, p95 4-7 ms
getPreferences	32.0k req/s, p95 5 ms
resolveHandle	23.9k req/s, p95 2 ms
describeRepo	39.0k req/s, p95 1 ms
getRecord	30.0k req/s, p95 5 ms
listRecords, limit 50	26.2k req/s, p95 6 ms
listMissingBlobs	27.6k req/s, p95 6 ms
sync JSON reads	24.1k-39.4k req/s, p95 1-4 ms
sync.getBlob	34.5k req/s, p95 1 ms
sync.getBlob HEAD	34.8k req/s, p95 1 ms
sync.getRepo	37.9k req/s, p95 below 1 ms
sync.getRepo HEAD	36.6k req/s, p95 below 1 ms

The separate permissioned-data scenario (just bench space) covers space discovery, management-policy reads, record writes and reads, writer/oplog enumeration, full repo CAR construction, referenced-blob enumeration, and blob serving. Keep it separate from public-repo benchmarks because the storage and commit formats are intentionally different.

The only pre-httpz route numbers currently available are the original narrow route set, run against commit 4841aec with the same four probes:

route	pre-httpz	httpz
/xrpc/_health	34.1k req/s, p95 3 ms	34.5k req/s, p95 3 ms
describeServer	38.4k req/s, p95 1 ms	39.3k req/s, p95 1 ms
getSession	19.9k req/s, p95 8 ms	20.8k req/s, p95 6 ms
listRecords, limit 50	19.8k req/s, p95 11 ms	22.4k req/s, p95 7 ms

traffic model #

For roughly 100 hosted accounts, useful benchmarks should mirror real traffic:

• clients call getSession, preferences, profiles, timelines, and proxied appview routes repeatedly while open.
• apps call createRecord, applyWrites, getRecord, and listRecords.
• media posts call uploadBlob before record creation.
• relays and tools call sync endpoints, repo export, and blob reads.
• the landing page reads residents, record counts, recent records, and profile blobs.

This traffic model is the basis for future route-mix benchmarks and per-route latency instrumentation. Keep those measurements separate from isolated storage microbenchmarks.

startup and deployment interruption #

Run just bench startup 5 10000 for a ReleaseSafe build (the production optimization mode) and five samples per scenario. Build time is excluded. tools/startup_bench.py measures subprocess launch to the first HTTP 200 health response with status: ok. It emits JSON with platform, binary SHA-256, individual samples and min/median/max. Polling adds up to roughly 5 ms plus HTTP request overhead; these are warm-filesystem runs, not cold VM boots.

The lanes separate fresh schema creation, empty existing-database restart, and history-shaped restart. The history fixture has 10,000 commits, 60,000 blocks with 4 KiB payloads, 10,000 sequence events with 4 KiB payloads, and 2,000 unversioned tree blocks which match neither records nor commits. It is a synthetic SQLite workload, not a signed repository or an API throughput benchmark. Increase the second argument to examine scaling.

For an existing local SQLite backup, run:

python3 tools/startup_bench.py --snapshot /path/to/backup.sqlite3 --runs 5

The source is opened read-only and copied with SQLite's backup API before every sample. The server only opens disposable copies. Runs bind to loopback, disable crawler notifications, discard inherited ZDS_* configuration, and clean up child processes and temporary data. Output exports stage durations, not server logs or account data. Use local backups rather than a production database path.

Startup logs report storage_ms, account_setup_ms, and http_init_ms. They measure storage initialization, account bootstrap/announcement setup, and HTTP server construction respectively. Storage also reports migrations_ms and sequence_ms to separate those two phases. The existing "listening" message precedes server.listen(); HTTP health remains the authoritative readiness measurement.

2026-09-05: index commit CID lookups #

The revision backfill runs on startup and matches NULL-revision blocks against commits by (did, cid). Without a matching index, unmatched historical tree blocks repeatedly scan an account's commit history. A non-unique composite index removes those scans without skipping backfill or integrity validation.

Five-run medians on macOS arm64, ReleaseSafe, warm filesystem. This initial fixture had block and commit history but no sequence-event payloads:

lane	before	indexed
fresh schema	22.00 ms	22.34 ms
empty restart	9.11 ms	9.33 ms
history restart	3147.59 ms	15.38 ms
old history snapshot, including index creation	—	29.01 ms

These fixture results are not production startup timings. Before the change, Fly logs showed about 5 seconds from process launch to the listening message. An external one-second health probe with a two-second request timeout observed 8 consecutive timeouts spanning roughly 17 seconds during the prior deployment; that includes machine replacement and proxy routing, not just process startup.

For production rollouts, record both process stage timings and an external health probe across replacement. Compare failure windows separately from healthy response latency. A single machine with a local SQLite volume has a deployment interruption even when application startup is fast; avoid extra restarts solely to publish benchmark notes.

sequence recovery #

The first indexed production deployment still recorded storage_ms=5709. The fixture improvement alone did not establish a production improvement. Read-only production probes then identified another scan: computing the next sequence via MAX over a union of every commit and event sequence. That query scanned the event table (including its large row storage). Computing MAX(seq) inside each branch first preserves empty-table and gap semantics while letting SQLite seek to each table's final integer primary key. A production read-only comparison returned 10914 for both queries, taking 556 ms and 1 ms respectively; these are individual warm query measurements, not full startup measurements.

The history fixture now includes 10,000 events with 4 KiB payloads to exercise this path. Regression tests cover empty histories, either history being ahead, gaps, and an empty commit table with retained events.

narrow revision backfill to missing revisions #

Production EXPLAIN QUERY PLAN showed SCAN repo_blocks for both backfill updates even after commit lookup was indexed. Unlike the read-only SELECT probes, UPDATE selected a full scan of payload-bearing block rows. A partial index on (did, cid) WHERE repo_rev IS NULL bounds subsequent backfill work to unversioned blocks. It is created after the legacy repo_rev column migration. Index creation still scans existing blocks once on upgrade; it does not remove or bypass migration work. The fixture now includes 4 KiB block payloads as well as events, exposing the cost of scanning payload-bearing tables.

With payloads in the fixture, five-run macOS ReleaseSafe median readiness fell from 33.86 ms (commit index only) to 9.22 ms (partial index plus per-table sequence maxima). Empty restarts measured 9.16 ms. These warm-cache results are distinct from the initial small-block comparison above.

September 2026: independent simplespace read/write access #

On macOS arm64, Zig 0.16.0, ReleaseFast, three just bench space 1000 runs before and after the API/storage update gave these median throughputs:

storage operation	before ops/s	after ops/s
listSpaces	56,055	54,399
listMembers	38,596	37,099
createRecord	7,389	7,224
getRecord	180,344	174,748
listRecords	44,609	43,957
getRepo	624	628

These sequential local samples show small changes (roughly 1–4% on most read paths), not a production capacity estimate. The member list now returns two additional flags and space configuration reads return independent policies. At 10,000 records, a single export probe measured 69.0 ops/s before and 68.6 after; createRecord measured 7,016 before and 7,319 after. Membership size is not scaled with record count in this fixture.

A ReleaseSafe startup rehearsal on a consistent production SQLite backup preserved account, session, OAuth, public block, permissioned record, space configuration, and membership state. The migration stage took 10 ms locally; first readiness was 336 ms and a subsequent restart reached readiness in 11 ms. These are individual warm-filesystem samples, not Fly replacement or network interruption measurements.