From e2b261b4b5761b00d3d9bf43d1ca6331c7537790 Mon Sep 17 00:00:00 2001 From: zzstoatzz Date: Tue, 4 Aug 2026 04:17:14 -0500 Subject: [PATCH] =?UTF-8?q?docs:=20handoff=20FINAL=20=E2=80=94=20soak=20v3?= =?UTF-8?q?=20passed=20(95=20min,=2019/19=20green,=20ship=20build)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 --- .../HANDOFF-2026-08-04-zio-backend.md | 28 +++++++++++-------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/docs/handoffs/HANDOFF-2026-08-04-zio-backend.md b/docs/handoffs/HANDOFF-2026-08-04-zio-backend.md index aa7bc33..0170ab8 100644 --- a/docs/handoffs/HANDOFF-2026-08-04-zio-backend.md +++ b/docs/handoffs/HANDOFF-2026-08-04-zio-backend.md @@ -1,7 +1,7 @@ # handoff 2026-08-04 — zlay on zio: seven fixes and a soak -> **STATUS: DRAFT — soak in progress.** The final section gets real numbers -> when the 90-minute window closes. Everything above it is finished work. +> **STATUS: FINAL.** Soak v3 passed: 95 minutes, 19/19 samples green, on +> the exact ship build. Details in the soak section. Continues [HANDOFF-2026-08-03-zio-backend.md](HANDOFF-2026-08-03-zio-backend.md). That handoff ended with: zlay runs on zio, holds ~47 connections, does not @@ -148,12 +148,18 @@ two disk-100% episodes without dropping connections. The window ended at (clean exit_code=0 of the VM — see machine notes), not due to any relay behavior. -**Soak v3** (same build, reprovisioned machine): the formal uninterrupted -90-minute certification. - -- [ ] 90-minute verdict: -- [ ] RSS start/end: -- [ ] metrics latency range: -- [ ] consumer frames delivered: -- [ ] recommended next: overnight soak, then operator conversation about a - canary +**Soak v3 — PASSED** (ship build `c383540` = zio `6790455` with the +detector gated off; reprovisioned machine; consumer attached throughout; +2026-08-04 07:40–09:15 UTC): + +- [x] **95 minutes, 19/19 samples green, zero crashes, zero assert trips** +- [x] connections 1,307–1,872 (normal fleet churn around ~1,831 seeded hosts) +- [x] RSS 900–1,057 MB oscillating, **no growth trend** — ~2.6x below + prod's flat 2.6 GB, and ~300 MB lighter than the detector builds +- [x] `/metrics` 0.034–0.405 s across all samples (never over half a second) +- [x] relay_seq 0 → 1,221,943 continuous; db queue depth 0 at every sample +- [x] consumer: ~812k frames / ~4.2 GB across two sessions at ~215–270 + frames/s, with a clean reconnect at the hourly session recycle +- [x] disk stable at 37–39% under the cron janitor +- [x] recommended next: operator canary (message drafted); the box keeps + running as unattended burn-in until its 24h lifecycle ends -- 2.51.2