Testing and receipts #

Stream's suites are the evidence behind semantic-parity.md. scripts/admit run executes all of them against one image and writes a receipt binding the results to that image's digest; a skipped suite blocks admission exactly like a failure. See gotchas.md for the traps that live in the harness rather than in Stream.

Everyday #

zig build test && zig build   # unit tests + exe
just simulator                # local fake atproto network on :7777
just run-sim                  # stream against it
just e2e                      # python wire checks
just bench                    # subsystem benchmarks

Contract suites #

Each runs real ReleaseSafe processes, offline.

just archive-contract              # archive XRPCs + resident manifest + pinned Go client
just plan-config-contract          # non-default planner limits/threshold
just cursor-lookback-contract      # v1/v2 replay-window behavior
just compaction-config-contract    # tombstone cap/rewrite workers, physical JSS
just retry-config-contract         # global/per-host retry admission + backoff
just subscribe-config-contract     # subscriber retention/cache/batching/slow controls
just subscribe-read-batch-contract # 5k-row filtered replay, exact cold pulls
just listener-contract             # public/debug isolation + disabled/legacy modes
just status-contract               # durable host rows + public HTTP view
just shutdown-contract             # drain, close 1001, hung listRepos, restart
just logging-contract              # formats, filtering, invalid configuration
just environment-contract          # JETSTREAM_* map, precedence, unknown rejection

environment-contract proves the hand-maintained Stream map, sorted unknown rejection, typed validation, CLI precedence, inspection, listener, storage, logging, and import authentication. It does not prove the map contains every flag in the pinned Go source; the gate requires a generated inventory comparison for that.

Oracles #

just oracle                        # lifecycle crash + RocksDB fault recovery matrix
just differential-oracle           # exact pinned upstream semantics, fully offline
just differential-oracle-multiseed # five upstream predicate-kill worlds
just powerloss-image               # one-time Linux NBD/ext4 tool-image bootstrap
just powerloss-oracle              # real power-cut recovery, offline after bootstrap

The power-loss tier is deliberately separate from ordinary unit and process tests. It cross-builds the production ReleaseSafe Linux binary, runs RocksDB and JSS on ext4 over a kernel NBD device, kills the block backend before the Stream process, and reconstructs the device only from writes acknowledged by FLUSH/FUA. The campaign covers eight lifecycle cuts, the two-crash cleanup guard, six delete-compaction cuts, and four timestamp-import rewrite cuts. Mutation cases use archives written by the production JSS writer and verify recovery through the production daemon and public V2 replay.

It requires privileged Linux containers and an available /dev/nbd3. The pinned Ubuntu tool image needs network access once for just powerloss-image; just powerloss-oracle itself refuses image pulls and runs from cached Zig, Python, simulator, and container inputs. The current recipe targets aarch64-linux-gnu, matching the development host used for its recorded receipt.

When offline suites cannot reach a failure #

Some failure modes are structurally unreachable in the fixture, and a green suite says nothing about them. The clearest example: the pinned simulator's repositories are ~4 KiB, so concurrent workers can never exhaust an in-flight budget that a single repository fits in. The batch submit/consume deadlock that ended experiment 4 survived 20 passing suites for exactly that reason, and four attempts to force it out of the simulator failed.

A real-network reproduction costs about €0.03. Provision a small cloud box, stream the image to it, and run against bsky.network with the pressure turned up — for that bug, 50 workers and --backfill-max-inflight-bytes=268435456 showed it in 90 seconds, and the before/after was unambiguous. Reach for this whenever a hypothesis involves memory pressure, worker contention, or repository sizes the simulator cannot produce.

just backfill-batch-contract pins the weaker property that a dispatch batch far larger than the job queue converges. Its docstring is explicit that it passes on the broken code; read it before treating it as coverage.

Dashboard and metrics #

Fully offline:

just dashboard-test
just process-metrics-contract
just http-metrics-contract
just dashboard-contract http://127.0.0.1:6008/metrics   # against a local candidate

process-metrics-contract also runs the dashboard family-presence check against the ReleaseSafe binary it launches. Family presence does not establish producer semantics, restart stability, or parity — that distinction is why the dashboard rows in the gate are still partial. grafana-dashboard.md records the checksum-pinned upstream source, the deterministic Stream adaptation, and the fail-closed metric contract.