From fd5a0a1116ade338582b106fd6ef8b5265c617a8 Mon Sep 17 00:00:00 2001 From: zzstoatzz Date: Thu, 30 Jul 2026 11:30:35 -0500 Subject: [PATCH] field log: CORRECTION -- the denominator was a different relay's database Every percentage reported on 2026-07-30 used 5,716,765 from the indigo relay at relay.waow.tech. Stream does not crawl that relay. The deployed command line is --relay-url=https://bsky.network, and bsky.network's own listHosts (which returns accountCount per host) sums to 22,845,221 accounts across 5,742 hosts. indigo tracks 2,551 hosts -- 44% -- so its count understated the target by roughly 4x. The feasibility investigation was right and got overridden. It estimated ~19.4M by correcting a raw 22.7M for accounts without crawlable repos, and the raw sum measured today is 22.85M. A days-old number derived from the correct relay was discarded for one pulled from the wrong relay and never reconciled, despite the two disagreeing by 3.4x in the same file. Corrected position at hour 59.5: 3,675,382 complete is 16-19%, not 63.6%. Remaining 15.7-19.2M repos at 48/s is 91-111h more against 60h of envelope left. Bootstrap does not finish in this envelope; even at a doubled 100/s it lands at 103-113h for bootstrap alone, before merge and cutover. Records the method failure: --relay-url was one docker inspect away for the whole run. Co-Authored-By: Claude Opus 5 (1M context) --- docs/full-network-experiment-2026-07.md | 34 +++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/docs/full-network-experiment-2026-07.md b/docs/full-network-experiment-2026-07.md index 52a211b..cd36e57 100644 --- a/docs/full-network-experiment-2026-07.md +++ b/docs/full-network-experiment-2026-07.md @@ -620,6 +620,40 @@ lexicons, captured without being enumerated in advance. Checked on one sealed segment, not all 4,818. +### CORRECTION: the denominator below is wrong — it is a different relay's database + +Everything in the next section, and every percentage reported during 2026-07-30, +used **5,716,765** from the indigo relay at `relay.waow.tech`. **Stream does not +crawl that relay.** The deployed command line is +`--relay-url=https://bsky.network`, and bsky.network's own +`com.atproto.sync.listHosts` — which returns `accountCount` per host — gives: + +| | hosts | accounts | +|---|---|---| +| **bsky.network** (what Stream crawls) | **5,742** | **22,845,221** (22,658,063 on active hosts) | +| indigo `relay.waow.tech` (what was used) | 2,551 | 5,716,765 | + +indigo tracks **44% of the hosts**. Using its count as "the network" understated +the target by roughly 4x. + +This also means the feasibility investigation was right and was overridden. It +estimated **~19.4M**, correcting a raw **22.7M** for accounts without crawlable +repos — and the raw sum measured directly today is 22.85M, matching. The number +that was days old and derived from the correct relay was discarded in favour of +one pulled from the wrong relay and never reconciled. + +**Corrected position at hour 59.5:** 3,675,382 complete is **16–19%**, not the +63.6% reported all day. Remaining is 15.7–19.2M repos, which at the observed +48 repos/s is **91–111 hours more** — against a 120-hour envelope with 60 hours +left. Bootstrap does not finish in this envelope. Even assuming the rate doubles +to 100/s as repositories shrink, it lands at 103–113 h for bootstrap alone, +before merge and cutover. + +**The method failure worth keeping:** the right denominator was one command away +(`docker inspect` the running container for `--relay-url`) for the entire run. +The wrong one was never checked against the project's own prior work, which +disagreed by 3.4x and was sitting in this same file. + ### What the 5,716,765 denominator does and does not mean Progress is quoted against ~5,716,765 repositories, taken from the indigo -- 2.54.0