zat.dev / stream
jetstream v2 in zig stream.waow.tech
Star
Fork 0
Watch
stream docs feature-surface.md
2.3 kB
Markdown
at zig-0.17

feature surface #

The detailed implemented surface, moved out of the README. As always, semantic-parity.md is the authority on what is proved, row by row; this lists what is built.

  • live feeds at /subscribe and /subscribe-v2, with collection and DID filters, cursor replay, runtime option updates, RFC 7692 permessage-deflate on v1, frozen v1 zstd, and fetchable dictionary-negotiated v2 zstd
  • a disk-backed historical archive through listSegments, getSegment, getBlock, and planBackfill. Sealed jss v1 segments are byte-compatible with upstream. Sealed headers, block envelopes, DID blooms, and collection summaries are resident and refresh atomically after seal, compaction, and timestamp rewrite; planning opens no segment files, and cold replay reads only selected compressed blocks
  • whole-network bootstrap with concurrent live capture and deterministic merge. Repository completion is coupled to ordinary archive-writer durability independently of the dispatch/checkpoint batch, so a restart does not replay work already committed. Startup resumes the existing active segment, truncating and fsyncing only a framing-torn suffix while failing closed on complete-frame corruption and sealed high-water checksum failure
  • Sync 1.1 commit verification — PLC key rotation, MST inversion, op-CID checks, replay protection, durable repository-chain state, and transparent authenticated whole-repo repair with ordered pending replay. The live scheduler implements Atmos-shaped 32-worker per-DID FIFO, bounded drop-oldest, and completion-order batching
  • RocksDB-backed metadata and archive publication coupled to segment fsync boundaries
  • tombstone-aware compaction and continuing repair of repositories that were unavailable or changed during bootstrap
  • crash-resumable timestamp imports with a durable RocksDB rule map, topology-preserving sealed-segment repair, and bearer-gated operator XRPC
  • operations views at /status — a summary field report plus hosts, accounts, collections, and segments tabs, with durable PDS diagnostics, per-repository reconstruction, and authoritative root verification against the PDS commit root. Prometheus metrics are on the optional debug listener at /metrics