From e0e40b8188ff65c33eee859bb94937acafa38d17 Mon Sep 17 00:00:00 2001 From: zzstoatzz Date: Thu, 13 Aug 2026 22:38:21 -0500 Subject: [PATCH] devlog 015: the service line Co-Authored-By: Claude Fable 5 --- devlog/015-the-service-line.md | 94 ++++++++++++++++++++++++++++++++++ 1 file changed, 94 insertions(+) create mode 100644 devlog/015-the-service-line.md diff --git a/devlog/015-the-service-line.md b/devlog/015-the-service-line.md new file mode 100644 index 0000000..1218040 --- /dev/null +++ b/devlog/015-the-service-line.md @@ -0,0 +1,94 @@ +# the service line + +on august 13th bluesky launched [Protocol Services](https://bsky.network) — +jetstream v2, official, with network replay. tens of billions of records +behind `planSnapshot`, live tails on `subscribeEvents`, TS and Go SDKs with +the announcement. it's jim calabro's design, and the launch is the moment +that design became a public contract. + +zat's stake in this is [stream.waow.tech](https://stream.waow.tech) — a +zig port of that design, built on zat, held to the upstream repo by a +differential oracle that drives jim's own client against it on every +admission run. port discipline means you re-pin deliberately when upstream +moves, and launch day was the move: the waypoint went to v0.2.0 the same +evening, the one server-side delta (end-to-end kinds filtering) ported and +gated before midnight. keeping pace with a release, same-day, with the +released client as referee — that's what the discipline is for. + +so, was this the biggest test of zat yet? yes — but not in the way a +benchmark is a test. the test was a week of operating a public firehose +under an instrument that refuses to be polite. + +## what the week actually tested + +[relay-eval](https://relay-eval.waow.tech) samples every sync source on the +network for five minutes out of every thirty and publishes the coverage. +it does not care that your unit tests pass. four zat releases in six days +were each bought by something it (or the operating of stream) surfaced: + +- **0.3.27** — reconnect backoff compounded on single-host consumers; a + relay that dropped every ~113s left a tail down a third of the time. +- **0.3.29** — a relay held the socket open and went silent for 2.5 + minutes; both streaming clients grew an idle watchdog, and the first + implementation was rewritten because `SO_RCVTIMEO` panics under + `std.Io.Threaded` in debug builds. +- **0.3.30** — a peer accepted a connection and never answered; + `receiveHead` waited forever. every phase of a fetch is now bounded. + +none of these are the kind of bug a library finds in itself. they are the +kind a library finds when it is the load-bearing wall of a service that +someone graphs in public. + +## the split + +the launch also settled an architecture question by example. bluesky +shipped `@bsky/sdk` split out of `@atproto/api`, "giving the protocol the +space it deserves as a universal technology." their line: protocol on one +side, bluesky services on the other. + +zat was blurring that exact line. `ArchiveBackfill` speaks +`network.bsky.jetstream.*` — a service API — and decodes jss v1, which is +jetstream's storage format. it vendored libzstd to do it. none of that is +atproto. + +so v0.4.0 draws the line the same way they did: `ArchiveBackfill`, the +jss decode, and the vendored zstd moved to +[zat.dev/jetstream](https://tangled.org/zat.dev/jetstream), the zig +jetstream service SDK, which depends on zat for everything protocol. +`JetstreamClient` (the v1 wire) stays for now — its consumers are real +and unbroken — but it's deprecated-in-home, and the `subscribeEvents` v2 +client will be born on the right side of the line. zat links no +compression library at all anymore. the protocol side got smaller, which +is how you know the cut was in the right place. + +## one more thing the week kept hand-rolling + +somewhere in all this, the same forty lines of RFC 3339 parsing appeared +for the fourth time — two byte-identical formatters in stream, a third +hiding in its server, a 45-line parser in the SDK example. the atproto +datetime string is not "time utils"; it's a pinned syntax profile, the +same family as `Tid` and `Nsid`, and zat already carried the interop +fixtures that define it — as test-only validation. + +v0.4.0 promotes that validator to `zat.Datetime`: parse to unix-epoch +microseconds (offsets normalized, the spec's year-zero floor enforced on +the *normalized* instant, so an offset can't smuggle one in), format to +the canonical `.ffffffZ`. the civil math is hinnant's days_from_civil, +because inventing calendar arithmetic in 2026 is how you get bugs with +anniversaries. the profile's divergences from general RFC 3339 — reject +`:60`, reject `-00:00`, upper-case `T`/`Z` only — were cross-checked +against rust's jiff and chrono before being committed to, so they're +decisions with receipts, not accidents. the interop fixtures now drive +the real parser, and four hand-rolled copies are one type. + +## the honest ending + +the eval is still finding things. connections through stream's front door +die at a low background rate that survived a day of instrumentation — +server exonerated, kernel exonerated, the proxy hop now under logging. +that hunt continues, and it will probably buy 0.4.1 or a stream fix or +both. that's the deal with instruments that refuse to be polite: they +don't stop when you'd like a quiet week. + +biggest test yet. passed where it counts, still running, and the design +being tested against was a gift — thanks, jim. -- 2.51.2