From 5a389adae2b28488318adc80a930a2badac15ee2 Mon Sep 17 00:00:00 2001 From: zzstoatzz Date: Thu, 30 Jul 2026 12:38:43 -0500 Subject: [PATCH] docs: verify the bloom fix cannot invalidate the existing archive Claimed a rebuilt binary would resume on the current data, reasoning from upstream's "not a format change" rather than from this codebase. That claim decides whether adopting the fix costs a rebuild or costs 1.37 TB, so it needed checking. segment_footer.zig writes an 8-byte region header carrying block count and per-bloom size ahead of the per-block blooms, and segment.zig blockBloomBlobs reads both back and indexes by multiplication. Nothing assumes 8,409 or the 4096 capacity anywhere in the read path. A segment sealed with fixed-size blooms records size=8409 and still reads; a right-sized one records its own size and reads identically, so both coexist in one archive. That is the equal-size-region invariant upstream describes, already implemented here. A rebuilt binary resumes on the current 3.8M repositories. Co-Authored-By: Claude Opus 5 (1M context) --- docs/jss-seal-spec.md | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/docs/jss-seal-spec.md b/docs/jss-seal-spec.md index 38b0728..70ba5f6 100644 --- a/docs/jss-seal-spec.md +++ b/docs/jss-seal-spec.md @@ -83,6 +83,31 @@ file describes how to *write* it. > scale. The machine is sized at 61 GB almost entirely for filters that are > ~14x larger than the one covering every DID in their own segment. > +> ### Verified: the fix does not invalidate the existing archive +> +> Upstream states the change is not a format change. Confirmed against *this* +> codebase rather than taking that on trust, because the answer decides whether +> adopting it costs a rebuild or costs the archive. +> +> `segment_footer.zig` writes an 8-byte region header before the per-block +> blooms holding **block count and per-bloom size**, and `segment.zig` +> `blockBloomBlobs` reads both back and indexes by multiplication: +> +> ```zig +> const count = readInt(u32, self.bytes, start); +> const size = readInt(u32, self.bytes, start + 4); +> const at = start + 8 + i * @as(u64, size); +> ``` +> +> Nothing assumes 8,409 or the 4096 capacity. A segment sealed with fixed-size +> blooms records `size = 8409` and still reads; a right-sized one records its own +> smaller size and reads the same way. **Old and new segments coexist in one +> archive**, which is exactly the equal-size-region invariant upstream describes, +> already implemented here. +> +> So a rebuilt binary resumes on the current archive. At hour 60 that is 1.37 TB +> and 3.8M repositories that would *not* be thrown away by adopting the fix. +> > ### What adopting it would take > > Upstream's `f02919c` changes `buildFooter` to drop its `maxEventsPerBlock` -- 2.51.2