zat.dev / stream
jetstream v2 in zig stream.waow.tech
Star
Fork 0
Watch
stream docs experiment-6-serving-receipt.md
3.6 kB
Markdown
at zig-0.17

experiment 6 — serving receipt #

Captured 2026-08-04 ~20:30Z, minutes after the phase 3 commit. Every claim below was verified against the running instance at capture time.

the phase commit #

20:10:47Z  merge-tail compaction: watermark 22976771240, 1 chunks,
           937/6479 segments rewritten, 90170067 rows dropped
20:11:43Z  merge discovery: skipped by configuration
20:11:44Z  phase -> steady_state
20:11:44Z  resuming from persisted cursor 32408688748

Bootstrap start to steady_state: 2026-07-28 05:01:47Z → 2026-08-04 20:11:44Z (~183h ≈ 7.6 days), including one deliberate cutover (unfetchable tail), one disk-full crash (reflink snapshot divergence), and a 6.6h merge-tail compaction.

proof of serving #

  1. Gate open. GET /xrpc/network.bsky.jetstream.listSegments → 200 with full segment metadata and cursor pagination (6,480 segments). Was 503 throughout bootstrap/merging. Note the route: network.bsky.jetstream.* — the jetstream.listSegments path used in earlier notes 404s.
  2. Read-back + decode. GET .../getSegment?name=seg_0000000000.jss (param is name, not segment — that one belongs to getBlock) returned 200 with all 277,192,887 bytes; stream inspect-segment on the download: magic jss0, state sealed, checksum 0x20f0665e7822a74c valid, 3,504,089 events / 858 blocks / seq [1, 3506746], 32 collection NSIDs enumerated. Size, checksum, counts all match the listSegments row exactly.
  3. Cleanup complete. /data/stream/backfill gone, 0 stale .tmp.
  4. /status: state live, phase steady_state, durable cursor 22,976,771,240 (~23.0B events durable), 16,300,003 repos total / 16,285,748 complete.

what was still pending at capture #

  • Live ingest. After the commit, main enters compact.steady.Compactor.rebuild — a synchronous full-archive fold rebuilding steady-compaction state before the consumer dials upstream (verified by stack: foldSegment ← Compactor.rebuild ← main, walking segments at ~25/min ⇒ ~4h). Until it finishes: upstream seq 0, events this run 0, no outbound connection. This is sequencing, not failure. A watcher fires when stream_upstream_seq > 0.
  • Replay. Cursor 32408688748 confirmed still replayable from bsky.network at capture (fresh wsprobe: 101 + binary frame). Window closes 2026-08-06 07:04:58Z; rebuild + connect fits with >24h margin.
  • Subscribe surface under live load — verify after replay starts.

addendum 22:40Z — live ingest + subscribe proven #

The Compactor.rebuild finished well under the ~4h extrapolation. At 22:35Z: upstream seq 32,408,817,638 (advancing from the 32,408,688,748 resume point), events this run 129,793, appended 334,817. /subscribe handshakes 101 and streams decoded Jetstream V2 JSON events (verified by raw websocket read: {"did":"did:plc:…","time_us":1785882887602279,"kind":"commit",…}). Every surface of a Jetstream V2 instance is now demonstrated: list, read-back, subscribe, live ingest. Remaining: replay catch-up to the firehose head.

honest footnotes #

  • merge discovery: skipped by configuration — the compose sets skip-merge-discovery, so DIDs that first appeared after the bootstrap cursor were not enqueued. Coverage is the bootstrap's 16.29M; new-account pickup begins with live ingest.
  • The kept/dropped ratio question from the handoff (§11) stands, softened by compaction's 90.17M dropped rows (85.2M sync, 3.2M record, 1.7M account).
  • One restart total during merging (disk full, self-healed); oom-kills all run: 1.