From 6e2055062eacf26b66115cf14554ad91c2b149c0 Mon Sep 17 00:00:00 2001 From: zzstoatzz Date: Thu, 30 Jul 2026 05:31:49 -0500 Subject: [PATCH] docs: subscribe protocol never explained v1 vs v2, or what "v1 quirk" means Every factual claim in this file verified out: both endpoints are served, the cursor magnitude constant is 1_000_000_000_000_000 in serve/server.zig, the v1 dict id 1612007021 and the v2 id 20260709 at 65,536 bytes are all real and the v2 pair is test-asserted down to a SHA-256. Nothing corrected; what was missing is a way in. The file uses "v1" and "v2" on nearly every line without ever saying how they relate or which Stream serves (both). Added a two-row table for the only real differences -- compression opt-in and the optional cursor envelope field -- and spelled out that "v1 quirk" means deliberately preserved bug-compatible behaviour, not a defect awaiting a fix, which is the whole point of the checklist at the end. Explains the cursor magnitude split, which reads as arbitrary: microsecond timestamps are already ~1.7e15 and sequence numbers start at 1, so the ranges cannot collide until one instance assigns 10^15 events. Records that v2_dictionary_id is derived from the embedded dictionary's own header, so the advertised ID and served bytes cannot disagree -- and that grepping for 20260709 finds nothing because Zig writes it 20_260_709. That digit-separator false negative cost a detour on this pass. Marks the upstream Go source paths and the docs/README.md 3.5 citation as upstream, consistent with the other two upstream specs. Co-Authored-By: Claude Opus 5 (1M context) --- docs/upstream-subscribe-protocol.md | 35 ++++++++++++++++++++++++++++- 1 file changed, 34 insertions(+), 1 deletion(-) diff --git a/docs/upstream-subscribe-protocol.md b/docs/upstream-subscribe-protocol.md index 4cc8dd2..10e737c 100644 --- a/docs/upstream-subscribe-protocol.md +++ b/docs/upstream-subscribe-protocol.md @@ -1,6 +1,39 @@ # jetstream v1 `/subscribe` wire protocol (from upstream v2 rewrite) -condensed from `bluesky-social/jetstream` `internal/subscribe/{doc,handler,filter,encoder,cursor,compress}.go`, docs/README.md §5. written 2026-07-08. +condensed from `bluesky-social/jetstream` +`internal/subscribe/{doc,handler,filter,encoder,cursor,compress}.go` and §5 of +upstream's `docs/README.md`. **Those are upstream Go paths — none of them are in +this repository, and the `docs/README.md` is upstream's, not this `docs/` +directory.** written 2026-07-08. + +## v1 and v2, since both appear throughout + +Stream serves **both** endpoints, `/subscribe` and `/subscribe-v2` +(`serve/server.zig`). They are the same event stream and the same JSON schema; +they differ only in compression and one envelope field: + +| | `/subscribe` (v1) | `/subscribe-v2` | +|---|---|---| +| compression opt-in | `compress=true` or `Socket-Encoding: zstd`; also negotiates `permessage-deflate` | `zstdDictionary=` only; silently declines `permessage-deflate` | +| `cursor` envelope field | omitted | optional | + +"v1 quirk" below therefore means *deliberately preserved bug-compatible +behaviour* — clients depend on it, so it is held rather than fixed. The +checklist at the end is the list of those. + +**Why the cursor magnitude split.** A cursor below 1e15 is read as a sequence +number and at or above it as microseconds since the epoch. That works because +microsecond timestamps are already ~1.7e15 while sequence numbers start at 1, so +the ranges cannot overlap until a single instance has assigned 10^15 events. The +constant is `1_000_000_000_000_000` in `serve/server.zig`, and both endpoints +use it. + +**The v2 dictionary ID is derived, not declared.** `v2_dictionary_id` is +computed from the embedded dictionary's own header +(`storage/zstd.zig`), so the advertised ID and the served bytes cannot +disagree. Tests pin the length (65,536), the ID (`20260709`), and a SHA-256 of +the artifact. Note Zig writes these with digit separators — `20_260_709` — so +grepping for `20260709` in the source finds nothing. ## connection & query params -- 2.51.2