stream #

a zig implementation of jetstream — the full-network archive and streaming service for atproto, designed and written by Jim Calabro. the archival design is jetstream V2's; this port carries it to zig, built on zat.

A live instance runs at stream.waow.tech — Jetstream V2-compatible feeds with the whole network archived behind them (~23B events, ~16.3M repos, sealed segments back to seq 1), live-current with the relay. Consumer docs, including endpoints and cursor semantics, are at stream.waow.tech/llms.txt.

what it does #

• live feeds: /subscribe (Jetstream v1-compatible JSON) and /xrpc/network.bsky.jetstream.subscribeEvents (proposal-0015 lexicon frames, sequence cursors), with collection/DID filters, cursor replay, and zstd
• the archive: listSegments, getSegment, getBlock, planSnapshot — gap-free recovery over sealed jss segments byte-compatible with upstream. A working consumer (plan → fetch → decode → sqlite) is examples/backfill-collection-sqlite.py; best suited to dense collections and bulk drains (block-granular retrieval)
• whole-network bootstrap with concurrent live capture, deterministic merge, and crash-safe resume
• Sync 1.1 commit verification and continuing repair of repositories that were unavailable or changed during bootstrap
• operations views at /status, Prometheus /metrics on the debug listener

The full implemented surface is in docs/feature-surface.md. Stream targets semantic parity with pinned Jetstream V2 commit d4dd2f0; docs/semantic-parity.md is the authority on what is proved, row by row, and no claim here overrides it.

develop #

zig build test && zig build   # unit tests + exe
just simulator                # local fake atproto network on :7777
just run-sim                  # stream against it

Suites, oracles, and the power-loss tier: docs/testing.md. The build pins Zig and all native deps (RocksDB, zstd, xxHash, zlib) — no system libraries required.

docs #

docs/ covers architecture, configuration, testing, and operations.