diff --git a/CHANGELOG.md b/CHANGELOG.md --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,48 @@ # changelog +## 0.3.25 + +The archive backfill release: consume a jetstream archive's full-history API +(stream.waow.tech and compatible instances) through the same handler the live +client uses. + +- **feat**: `zat.ArchiveBackfill` — planBackfill → getBlock/getSegment → zstd + decompress → jss v1 columnar decode → `JetstreamEvent`s through the existing + `onEvent` handler; a consumer's live-tail code replays history unmodified. + Delivery is at-least-once, in plan order. Includes bounded per-fetch retries + (5xx/429/transport errors), `start_after`/`Result.position` resume so a + transient failure never discards a multi-GB run, opt-in whole-segment + checksum verification (xxh3 self-checksum cross-checked against the plan), + `max_blocks` for bounded exploratory runs, and opt-in `concurrency` — an + ordered FIFO fetch window measured 7.2x faster than serial on WAN block + fetches, with delivery order unchanged. `replaySegment` decodes a sealed + jss segment already in memory. The module doc records the measured scope + boundary: block-granular reads are ~1,300x amplified for sparse scattered + collections; those belong to a collection directory + listRecords. + +- **feat**: vendored libzstd v1.5.7, decode-only subset (same pin as stream; + 10 C files, pinned by url+hash, no system library, cross-compiles with + `-Dtarget`). Zig std's zstd decompressor measured ~20x slower than libzstd + on real jss blocks (457ms → 19ms for a 58-block segment — ahead of the Go + reference implementation at 37ms; atproto-bench `bench-jss` lane) and its + checksum verification is unimplemented, so frame content checksums went + unchecked. Every jss block's content checksum is now verified during + decompression. + +- **feat**: `JetstreamClient` primary failback — after a connection that + delivered events drops, the next attempt starts at `hosts[0]` instead of + sticking to the fallback rotation until process restart. + +- **feat**: `JetstreamClient.idle_timeout_ms` — fail over from a host that + holds the socket open but goes silent, which TCP keepalive never fires on. + +## 0.3.24 + +CI and examples housekeeping: docs deployment hardening (Wisp retries, +credential fixes, PDS-recovery retry), a Streamplace chat history replay +example and its session-replay fix, and tolerating undeclared fields in DoH +safety-check responses. + ## 0.3.23 Three transport failures found in stream's whole-network backfill, all of them