essentially stats-backfill without the measurement walk: repos are never parsed, just streamed from getRepo straight into car files. progress lives in the same style of sqlite db, so runs are resumable and monitorable, and basic per-repo size stats (wire bytes, decoded bytes, content-encoding) are still recorded.

heads up: THIS WILL BACKFILL THE WHOLE NETWORK. it's not meant for running more than once, which would be wasteful for both you and the network.

output layout #

out-dir/
  .partial/                  staging; wiped at startup, do not touch
  plc/<cc>/did:plc:….car     <cc> = first two chars of the plc id (1024 buckets)
  web/did:web:….car

non-plc/web did methods are rejected (atproto allows no others).

a .car file at its final path is always complete: downloads stream into .partial/ and are fsync'd + renamed into place only on success. when a higher rev for a did shows up on another host, the file is re-fetched and atomically replaced (highest-listRepos-rev-wins, same as stats-backfill).

only run one instance per out-dir (startup wipes .partial/).

host backpressure #

per-host fetch concurrency starts at --per-pds-limit (default 10). two consecutive overload-ish failures (429 / 5xx / connect / timeout) halve it: 10 → 5 → 2 → 1. only when a host keeps failing at concurrency 1 does it go into cooldown (Retry-After-aware). reduced limits are persisted per host, so restarts don't hammer a struggling host at full blast again.