diff --git a/docs/upstream-subscribe-protocol.md b/docs/upstream-subscribe-protocol.md index 4cc8dd2..10e737c 100644 --- a/docs/upstream-subscribe-protocol.md +++ b/docs/upstream-subscribe-protocol.md @@ -1,6 +1,39 @@ # jetstream v1 `/subscribe` wire protocol (from upstream v2 rewrite) -condensed from `bluesky-social/jetstream` `internal/subscribe/{doc,handler,filter,encoder,cursor,compress}.go`, docs/README.md §5. written 2026-07-08. +condensed from `bluesky-social/jetstream` +`internal/subscribe/{doc,handler,filter,encoder,cursor,compress}.go` and §5 of +upstream's `docs/README.md`. **Those are upstream Go paths — none of them are in +this repository, and the `docs/README.md` is upstream's, not this `docs/` +directory.** written 2026-07-08. + +## v1 and v2, since both appear throughout + +Stream serves **both** endpoints, `/subscribe` and `/subscribe-v2` +(`serve/server.zig`). They are the same event stream and the same JSON schema; +they differ only in compression and one envelope field: + +| | `/subscribe` (v1) | `/subscribe-v2` | +|---|---|---| +| compression opt-in | `compress=true` or `Socket-Encoding: zstd`; also negotiates `permessage-deflate` | `zstdDictionary=` only; silently declines `permessage-deflate` | +| `cursor` envelope field | omitted | optional | + +"v1 quirk" below therefore means *deliberately preserved bug-compatible +behaviour* — clients depend on it, so it is held rather than fixed. The +checklist at the end is the list of those. + +**Why the cursor magnitude split.** A cursor below 1e15 is read as a sequence +number and at or above it as microseconds since the epoch. That works because +microsecond timestamps are already ~1.7e15 while sequence numbers start at 1, so +the ranges cannot overlap until a single instance has assigned 10^15 events. The +constant is `1_000_000_000_000_000` in `serve/server.zig`, and both endpoints +use it. + +**The v2 dictionary ID is derived, not declared.** `v2_dictionary_id` is +computed from the embedded dictionary's own header +(`storage/zstd.zig`), so the advertised ID and the served bytes cannot +disagree. Tests pin the length (65,536), the ID (`20260709`), and a SHA-256 of +the artifact. Note Zig writes these with digit separators — `20_260_709` — so +grepping for `20260709` in the source finds nothing. ## connection & query params