# stream

a zig implementation of [jetstream](https://github.com/bluesky-social/jetstream) — the full-network archive and streaming service for atproto, designed and written by [Jim Calabro](https://calabro.io). the archival design is jetstream V2's; this port carries it to zig, built on [zat](https://tangled.org/zat.dev/zat).

A live instance runs at **[stream.waow.tech](https://stream.waow.tech)** —
Jetstream V2-compatible feeds with the whole network archived behind them
(~23B events, ~16.3M repos, sealed segments back to seq 1), live-current with
the relay. Consumer docs, including endpoints and cursor semantics, are at
[stream.waow.tech/llms.txt](https://stream.waow.tech/llms.txt).

## what it does

- **live feeds**: `/subscribe` (Jetstream v1-compatible JSON) and
  `/xrpc/network.bsky.jetstream.subscribeEvents` (proposal-0015 lexicon frames, sequence cursors), with collection/DID
  filters, cursor replay, and zstd
- **the archive**: `listSegments`, `getSegment`, `getBlock`, `planSnapshot` —
  gap-free recovery over sealed `jss` segments byte-compatible with upstream.
  A working consumer (plan → fetch → decode → sqlite) is
  [`examples/backfill-collection-sqlite.py`](examples/backfill-collection-sqlite.py);
  best suited to dense collections and bulk drains (block-granular retrieval)
- **whole-network bootstrap** with concurrent live capture, deterministic
  merge, and crash-safe resume
- **Sync 1.1 commit verification** and continuing repair of repositories that
  were unavailable or changed during bootstrap
- **operations views** at `/status`, Prometheus `/metrics` on the debug
  listener

The full implemented surface is in
[`docs/feature-surface.md`](docs/feature-surface.md). Stream targets semantic
parity with pinned Jetstream V2 commit `d4dd2f0`;
[`docs/semantic-parity.md`](docs/semantic-parity.md) is the authority on what
is proved, row by row, and no claim here overrides it.

## develop

```
zig build test && zig build   # unit tests + exe
just simulator                # local fake atproto network on :7777
just run-sim                  # stream against it
```

Suites, oracles, and the power-loss tier: [`docs/testing.md`](docs/testing.md).
The build pins Zig and all native deps (RocksDB, zstd, xxHash, zlib) — no
system libraries required.

## docs

[docs/](docs/) covers architecture, configuration, testing, and operations.