feat(jetstream): rev-gated multi-feed ingestion — escape bsky.network quotas master
The AppView now consumes N Jetstream feeds carrying the same repos: the public bsky.network Jetstream plus our self-hosted relay+Jetstream pair (tidepool stack) that crawls tdpl.io + pds.coves.me with no per-host account quotas. Feeds are internally ordered but skewed by hours, so a lagging feed replays a repo's history AFTER newer events were applied — resurrecting deleted comments/votes, regressing edits, re-subscribing unsubscribed users. The existing time_us recency guards cannot catch this: each feed stamps its own emission time, so a stale copy arrives with a NEWER timestamp than the state it would clobber. The keystone is rev-gating: every commit event carries rev, the repo's monotonic TID. jetstream_record_revs (migration 033, COLLATE "C") stores the last APPLIED rev per record URI; create/update/delete apply only if strictly newer. Equal rev = duplicate replay (no-op, subsumes existing duplicate handling); the gate row survives hard deletes as the tombstone that rejects stale creates. One rule, no heuristics, no CID comparisons. Changes: - rev_gate.go: conditional-upsert gate primitives + RevGate + applyGated (transactional claim held across apply; same-URI handlers serialize on the gate row lock, apply failure rolls back un-advanced for redrive) - posts/comments/votes: gate as first statement of existing transactions; delete paths restructured gate-first so the tombstone commits atomically with the deletion (closes the not-found-delete vs concurrent-create race) - users/communities/aggregators: injected RevGate; deletes tombstone even for never-indexed records; comments re-apply genuinely-newer same-rkey re-creates on active rows; SubscribeWithCount conflict path updates record_uri/cid last-write-wins (cross-rkey redriven-delete safety) - feeds.go: JETSTREAM_FEEDS="bsky=…;self=…" replaces six per-consumer URL env vars (now fatal at boot with migration hint); per-consumer collection filters derived in code via WantedCollections (unknown name = fatal, no more filterless whole-firehose subscriptions); "bsky" feed keeps legacy consumer names so live cursors carry over; @self consumers live-tail - fail-closed boot checks: multi-feed refuses ungated consumers (RevGated()), missing JETSTREAM_FEEDS fatal outside dev, warning when no primary feed key is configured - fixes latent drift: community.block was never in the subscribed collections; users consumer subscribed to the entire firehose in prod - tests: adversarial cross-feed interleavings (zombie create, stale update clobber, phantom vote, delete-before-create tombstones for votes AND posts, subscription zombie, profile stale-replay, gate semantics) + feeds parsing suite; Makefile test target runs packages sequentially (-p 1) so the integration suite's unscoped table wipes can't race other packages on the shared test DB Known limitations (documented in code): identity events carry no rev so handle changes are not cross-feed ordered; posts/votes active-row same- rkey re-creates after a dead-lettered delete keep the old content. Deploy notes: migration 033 auto-runs at boot. Prod compose sets JETSTREAM_FEEDS with self=ws://tidepool-prod-jetstream:8080 (relay + Jetstream deployed 2026-07-17). Expect "rev-gate: skipped stale" log lines for lagging bsky copies — that is the system working. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>