From c1f99c2c21b8331c51896b2fab259246ac8bf5e8 Mon Sep 17 00:00:00 2001 From: zzstoatzz Date: Mon, 2 Mar 2026 05:02:21 -0600 Subject: [PATCH] =?UTF-8?q?docs:=20rewrite=20readme=20=E2=80=94=20drop=20s?= =?UTF-8?q?tale=20numbers=20table,=20fix=20config,=20link=20to=20spec?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit the hard-coded metrics table was already stale and duplicated what the public grafana dashboard shows. replaced the endpoints table with a link to the sync spec. fixed RELAY_HTTP_PORT → RELAY_METRICS_PORT and added RESOLVER_THREADS + VALIDATOR_CACHE_SIZE. Co-Authored-By: Claude Opus 4.6 --- README.md | 41 +++++++++++++++-------------------------- 1 file changed, 15 insertions(+), 26 deletions(-) diff --git a/README.md b/README.md index 4c5168e..fc8e276 100644 --- a/README.md +++ b/README.md @@ -1,31 +1,22 @@ # zlay -an [AT Protocol](https://atproto.com/) relay in zig. subscribes to every PDS on the network, verifies commit signatures, and serves the merged event stream to downstream consumers via `com.atproto.sync.subscribeRepos`. +an [AT Protocol](https://atproto.com/) relay in zig. subscribes to every PDS on the network, verifies commit signatures, and serves the merged event stream to downstream consumers. -**live instance**: [zlay.waow.tech](https://zlay.waow.tech/_health) — [metrics](https://zlay-metrics.waow.tech) +**live instance**: [zlay.waow.tech](https://zlay.waow.tech/_health) — [metrics dashboard](https://zlay-metrics.waow.tech) ## design -- **direct PDS crawl** — the bootstrap relay (`bsky.network`) is called once at startup for the host list via `listHosts`, then all data flows directly from each PDS. +- **direct PDS crawl** — the bootstrap relay is called once at startup for the host list via `listHosts`, then all data flows directly from each PDS. -- **optimistic signature validation** — on signing key cache miss, the frame passes through immediately and the DID is queued for background resolution. the first commit from an unknown account is unvalidated; all subsequent commits are verified against the cached key. >99.9% cache hit rate after warmup. +- **optimistic signature validation** — on signing key cache miss, the frame passes through immediately and the DID is queued for background resolution. all subsequent commits are verified against the cached key. the cache caps at a configurable size and evicts the oldest 10% by resolve time when full. -- **inline collection index** — indexes `(DID, collection)` pairs directly in the event processing pipeline using [RocksDB](https://rocksdb.org/) with two column families: `rbc` for collection-to-DID lookups and `cbr` for DID-to-collection cleanup. serves `listReposByCollection` from the relay process — no sidecar. the index design draws on [fig](https://tangled.org/microcosm.blue)'s work on [lightrail](https://tangled.org/microcosm.blue/lightrail), which uses adjacent keys from CAR slices to enumerate collections. +- **inline collection index** — indexes `(DID, collection)` pairs in the event processing pipeline using RocksDB. serves `listReposByCollection` from the relay process — no sidecar. the index design draws on [fig](https://tangled.org/microcosm.blue)'s work on [lightrail](https://tangled.org/microcosm.blue/lightrail). -- **one OS thread per PDS** — predictable memory, no garbage collector. ~2,750 threads is fine; most are blocked on websocket reads. thread stacks are set to 2 MB (zig's default is 16 MB). +- **one OS thread per PDS** — predictable memory, no garbage collector. thread stacks are set to 2 MB (zig's default is 16 MB). -## endpoints +## spec compliance -| endpoint | method | -|---|---| -| `com.atproto.sync.subscribeRepos` | WebSocket | -| `com.atproto.sync.listRepos` | GET | -| `com.atproto.sync.getRepoStatus` | GET | -| `com.atproto.sync.getLatestCommit` | GET | -| `com.atproto.sync.listReposByCollection` | GET | -| `com.atproto.sync.listHosts` | GET | -| `com.atproto.sync.getHostStatus` | GET | -| `com.atproto.sync.requestCrawl` | POST | +implements the [AT Protocol sync spec](https://atproto.com/specs/sync) — `subscribeRepos`, `listRepos`, `getRepoStatus`, `getLatestCommit`, `listReposByCollection`, `listHosts`, `getHostStatus`, and `requestCrawl`. ## dependencies @@ -34,7 +25,7 @@ an [AT Protocol](https://atproto.com/) relay in zig. subscribes to every PDS on | [zat](https://tangled.org/zzstoatzz.io/zat) | AT Protocol primitives (CBOR, CAR, signatures, DID resolution) | | [websocket.zig](https://github.com/nicholasgasior/websocket.zig) | WebSocket client/server | | [pg.zig](https://github.com/karlseguin/pg.zig) | PostgreSQL driver | -| [rocksdb-zig](https://github.com/Syndica/rocksdb-zig) | [RocksDB](https://rocksdb.org/) bindings | +| [rocksdb-zig](https://github.com/Syndica/rocksdb-zig) | RocksDB bindings | ## build @@ -50,21 +41,19 @@ zig build -Doptimize=ReleaseSafe # release build | variable | default | description | |---|---|---| -| `RELAY_PORT` | `3000` | WebSocket firehose port | -| `RELAY_HTTP_PORT` | `3001` | HTTP API port | +| `RELAY_PORT` | `3000` | firehose + API port | +| `RELAY_METRICS_PORT` | `3001` | prometheus metrics port | | `RELAY_UPSTREAM` | `bsky.network` | bootstrap relay for initial host list | | `RELAY_DATA_DIR` | `data/events` | event log storage | | `RELAY_RETENTION_HOURS` | `72` | event retention window | | `COLLECTION_INDEX_DIR` | `data/collection-index` | RocksDB collection index path | | `DATABASE_URL` | — | PostgreSQL connection string | | `RELAY_ADMIN_PASSWORD` | — | bearer token for admin endpoints | +| `RESOLVER_THREADS` | `4` | background DID resolution threads | +| `VALIDATOR_CACHE_SIZE` | `500000` | max cached signing keys before eviction | see [docs/deployment.md](docs/deployment.md) for production deployment and [docs/backfill.md](docs/backfill.md) for collection index backfill. -## numbers +## license -| metric | value | -|---|---| -| connected PDS hosts | ~2,750 | -| memory | ~2.9 GiB | -| throughput | ~600 events/sec typical | +MIT -- 2.51.2