From e2f69c0da2dbdd8ab2c0c0b122be0426730c15c6 Mon Sep 17 00:00:00 2001 From: zzstoatzz Date: Tue, 14 Jul 2026 01:51:06 -0500 Subject: [PATCH] record failed full-network experiment teardown --- docs/full-network-experiment-2026-07.md | 31 +++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/docs/full-network-experiment-2026-07.md b/docs/full-network-experiment-2026-07.md index c2ea2e4..29865e7 100644 --- a/docs/full-network-experiment-2026-07.md +++ b/docs/full-network-experiment-2026-07.md @@ -85,3 +85,34 @@ The 24-hour monitor should measure from this clean baseline while retaining the original cost deadline. Any later rollout must be called out because it changes the interpretation of repository completion and throughput. +## Termination and disposition + +The experiment was terminated early on 14 July 2026 after the live crawl found +repositories larger than Stream's hard-coded 512 MiB `getRepo` CAR limit. That +limit has no equivalent in upstream Jetstream V2: upstream streams the HTTP +response into its CAR reader, retains decoded blocks in an in-memory block +store, checks that the full MST is complete, and imposes a download deadline +rather than a repository-size exclusion. + +Consequently this run did not demonstrate full-network backfill. The rejected +repositories made the partial archive semantically invalid for that claim, so +the archive was not retained. The following experiment-only Hetzner resources +were deleted and provider inventory was checked afterward: + +- server `stream-experiment` (`150777708`); +- volume `stream-experiment-data` (`106359203`, 4 TB); and +- firewall `stream-experiment-fw` (`11304930`). + +The final public snapshot before deletion showed upstream sequence 853,602,603, +four active backfill workers, eight queued jobs, 751,123,595 bytes of transient +in-flight allocation, zero completed durability pages, and zero failed +durability pages. The zero failure count did not contradict the observed +oversized-repository errors: results for the first 1,000-repository page had not +yet crossed the page durability boundary. + +No replacement historical-bootstrap deployment is permitted until Stream has +semantic equivalents for the upstream backfill path. At minimum, that means no +arbitrary repository-size exclusion; incremental CAR consumption; complete-MST +verification before completion; durable per-repository failure and retry state; +and restart-safe progress whose completion semantics match upstream. Passing +shape-compatible endpoint tests alone is not sufficient evidence. -- 2.51.2