From d83eb8f9e6a3a4e080476bdd81c792dfa521276e Mon Sep 17 00:00:00 2001 From: zzstoatzz Date: Thu, 30 Jul 2026 07:31:40 -0500 Subject: [PATCH] docs: timestamp import never explained which timestamp, or why that is safe The opening sentence distinguishes "the timestamp presented to subscribers" from "the immutable witness timestamp" and then never returns to it, though the entire design turns on that split. Added a table naming both (display/time_us, changed by an import; witness/witnessed_at, never) and what each is used for. The part worth stating is why the separation exists rather than that it does: witnessed_at defines segment ordering and is what a ?cursor= replay resolves against, so if an import could move it, every sealed segment header range and every timestamp cursor would become wrong and the "segments sort in creation order, and that order is time order" invariant in invariants.md would break. An import is a presentation layer over immutable history, not a rewrite. Also named the operator use case -- backfilled or migrated content whose real creation time predates when this instance first saw it. Verified rather than assumed: both flags exist in runtime/cli.zig and both XRPC routes are in serve/server.zig. Noted that the routes are matched as full literal /xrpc/... paths, so grepping for the bare NSID finds nothing -- which is how I briefly convinced myself they were missing. Co-Authored-By: Claude Opus 5 (1M context) --- docs/timestamp-import.md | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/docs/timestamp-import.md b/docs/timestamp-import.md index 3975931..0ad8e11 100644 --- a/docs/timestamp-import.md +++ b/docs/timestamp-import.md @@ -4,6 +4,33 @@ Stream follows Jetstream V2's operator timestamp-import design. An import changes the timestamp presented to subscribers without changing the immutable witness timestamp used for archive ranges and timestamp cursors. +## What this is for, and the one distinction the whole design turns on + +Records carry **two** timestamps, and an import moves only one of them: + +| | changed by an import? | used for | +|---|---|---| +| **display timestamp** (`time_us` on the wire) | **yes** | what subscribers see | +| **witness timestamp** (`witnessed_at`) | **never** | archive ranges, timestamp cursors, segment header min/max | + +That separation is why an import is safe at all. `witnessed_at` is when *this +instance* observed the event, so it defines segment ordering and is what a +`?cursor=` replay resolves against. If an import could move it, +every sealed segment's header range and every timestamp cursor would become a +lie, and the "segment files sort in creation order, and that order is time +order" invariant would break. So imports are a **presentation** layer over +immutable history, not a rewrite of it. + +The operator use case is backfilled or migrated content whose true creation time +is older than when this instance first saw it — without an import, a repository +imported today would present all its history as today's events. + +Verified 2026-07-30: `--timestamp-import-dir` and `--timestamp-import-token` +both exist in `runtime/cli.zig`, and `/xrpc/network.bsky.jetstream.importTimestamps` +and `/xrpc/network.bsky.jetstream.getImportStatus` are both routed in +`serve/server.zig`. (Those are matched as full literal paths, so grepping the +source for the bare NSID finds nothing.) + ## Source format The source is a plain, seekable RFC 4180 CSV. `uri` and `timestamp` are -- 2.51.2