# getRepo notes

Context: on 2026-06-29, operators called out `com.atproto.sync.getRepo` as an expensive route for PDS hosts. The main concerns were compression cost, large repo export reads, avoiding disclosure of stale/deleted blocks, and isolating backup/backfill traffic from user-facing work.

ZDS is directly relevant because it serves `com.atproto.sync.getRepo` from `src/atproto/sync.zig`.

Current shape:

- ZDS serves raw CAR bytes and does not appear to add application-layer gzip, so it avoids the Node compression issue described in the operator thread.
- As of the 2026-06-29 getRepo pass, full repo export starts at the latest commit block and uses `zat.mst.collectReachableBlocks` to include only current MST and record blocks reachable from the commit's `data` CID.
- Incremental export with `since` filters by `repo_rev`.
- Repo writes already use lazy MST loading backed by `repo_blocks`, so ZDS is closer to a Hubble-style block/CAR-serving shape than an eager full-repo rebuild path.
- File-backed exports use a dedicated read-only SQLite connection and WAL
  transaction. The root and reachable blocks come from one coherent snapshot;
  CAR encoding happens after that snapshot closes. Account, OAuth, and write
  operations continue on the primary connection instead of waiting behind repo
  traversal and encoding.
- `subscribeRepos` has a connection cap. Full `getRepo` now has its own lightweight route-local concurrency cap so backup/backfill exports cannot consume unbounded handler slots. Operators can tune it with `ZDS_MAX_CONCURRENT_REPO_EXPORTS`; the default is `4`, matching Tranquil's current full-export default.

Open questions:

- Should `HEAD com.atproto.sync.getRepo` stay semantically cheap, or should it compute/export-sized metadata? Today it is treated as a probe and still shares the same handler path.
- Should `getRepo` backpressure eventually become per-account/IP rate limiting in addition to process-local concurrency?
- Should the benchmark suite keep a historical range-scan comparison around, now that production full export is reachability-based?

Done in this pass:

1. Adopted `zat` 0.3.9 for `collectReachableBlocks`.
2. Added a regression test for update/delete followed by full `getRepo`; stale record CIDs are excluded from full export.
3. Changed full export from whole-DID `repo_blocks` scan to current-root reachability.
4. Added lightweight route backpressure for full `getRepo`, configurable with `ZDS_MAX_CONCURRENT_REPO_EXPORTS`.
5. Ran `just bench repo-size`; reachable full export measured about 1.4 ms for 100 records, 16.8 ms for 1,000 records, and 95.7 ms for 5,000 records on the local synthetic benchmark.
6. Added `just bench get-repo` as the focused storage-pressure lane for this route. It measures full export construction, one-commit `since` export construction, and concurrent full export latency against one seeded repo.
7. Added account-read and write latency to the focused benchmark after
   production traffic demonstrated that export throughput alone missed
   process-wide database contention.
8. Moved file-backed full and incremental exports to independent read
   transactions. This follows the isolation boundary used by the reference
   PDS's actor-store reader and Tranquil's separate block store while retaining
   ZDS's SQLite/WAL architecture.

Validation on 2026-07-30:

- In the local 10k-record, four-export comparison, the previous implementation
  delayed an account lookup by 325 ms and a write by 336 ms. The isolated
  implementation kept their maxima to 0.08 ms and 1.92 ms respectively.
- On a disposable Fly `shared-cpu-1x` machine with 1 GB RAM, eight concurrent
  exports of a 10k-record repo completed without failure. Account lookups
  remained below 20 ms and writes below 44 ms even while the single CPU was
  saturated.

Suggested next pass:

1. Add a larger deliberate `just bench get-repo 100000 4 1` run when we want stress numbers comparable to large public repos.
2. Add a cross-PDS HTTP/export comparison in `atproto-bench` once the ZDS-local storage seam is stable.