From 2c3a268642e7aeb2d8f91b4bb9d6151920c8c7ea Mon Sep 17 00:00:00 2001 From: Bretton Date: Fri, 7 Aug 2026 15:22:24 -0700 Subject: [PATCH] docs: author-owned-posts PRD rev 2.1 + build-loop plan/tracker Design: posts flip to author-repo records (social.coves.community.postv2) with community-signed acceptance/removal records; per-(community,post) admissions state. Externally reviewed (Codex gpt-5.6-sol, 2026-08-05). Loop: 8 PR-sized tasks, /tdd -> /second-opinion -> /fix-pr -> make ci -> --no-ff merge per task. Co-Authored-By: Claude Fable 5 --- docs/PRD_AUTHOR_OWNED_POSTS.md | 589 +++++++++++++++++++++++++++++++++ loop_state.md | 39 +++ plan.md | 204 ++++++++++++ 3 files changed, 832 insertions(+) create mode 100644 docs/PRD_AUTHOR_OWNED_POSTS.md create mode 100644 loop_state.md create mode 100644 plan.md diff --git a/docs/PRD_AUTHOR_OWNED_POSTS.md b/docs/PRD_AUTHOR_OWNED_POSTS.md new file mode 100644 index 0000000..b614865 --- /dev/null +++ b/docs/PRD_AUTHOR_OWNED_POSTS.md @@ -0,0 +1,589 @@ +# PRD: Author-Owned Posts + Community Acceptance Records + +**Status: DRAFT rev 2 — design agreed 2026-08-01; revised 2026-08-05 after an +external design review (OpenAI Codex `gpt-5.6-sol`, high effort, full repo +access). The review confirmed the two-record trust split and forced five +structural changes: admissions became per-(community, post) relational state +instead of columns on `posts` (§6.1), cross-record ordering got a subject-scoped +watermark and atomic community-repo commits (§5.2), a `rejected` decision state +now actually exists (§6.1), removal is defined as terminal across author edits +(§5.5), and the spec stops claiming today's write path enforces bans — it does +not; ban enforcement is new scope (§4.1). +Rev 2.1 (2026-08-07): corrected for the fact that the lexicons ARE published — +the flipped post record moves to a NEW NSID (`social.coves.community.postv2`) instead +of breaking `social.coves.community.post` in place (§3.0/§3.1); lexicon +details audited against the atProto Lexicon spec, record-key spec, and the +draft Lexicon style guide (bluesky-social/atproto discussion #4245) — open +value sets confirmed, deterministic rkeys specified per the record-key spec's +"(transformed) AT URI" pattern.** + +**Supersedes** the write-path architecture in `docs/federation-prd.md`: that +document solves cross-instance posting by service-auth-forwarding the write to +the community's host, which still signs the post into the community repo. This +PRD removes the need for any server to write posts on behalf of another +server's users. The service-auth *transport* from federation-prd is retained +for the notify endpoint, but its specification was hypothetical and is +re-specified concretely in §7. + +--- + +## 1. Problem + +Posts are the only user-authored record type that lives in the **community's** +repo. Comments (`internal/core/comments/comment.go:14`) and votes +(`internal/core/votes/vote.go:8`) already live in the **author's** repo and are +indexed from the firehose. Posts are write-forwarded: the AppView validates, +then uses the community's stored PDS credentials to sign the author's words +into the community repo (`internal/core/posts/service.go` `CreatePost`). + +This causes three structural problems: + +1. **Cross-server posting is impossible.** A user on Coves-canonical cannot + post into a community hosted on Coves-selfhosted: their server doesn't hold + that community's keys, and never should. +2. **Authorship is unverifiable.** The `author` field inside the post record is + an unsigned claim by whoever holds the community keys. Any community host + can fabricate posts by any user. +3. **Forks orphan the interaction graph.** A community fork is a new DID, so it + must re-sign every post → new AT-URIs/CIDs → every comment and vote in every + user repo still points at `at://old-community-did/...`. A fork today gets + post husks with no comments and no scores. + +## 2. Design summary + +**The user's speech is signed by the user; the community's acceptance of that +speech is signed by the community. Two facts, two records, two repos.** + +- The post record moves to the **author's repo** (same placement as comments + and votes), as the new lexicon `social.coves.community.postv2` (§3.0 — the old + published NSID is deprecated, not broken). The repo signature becomes the + authorship anchor; there is no in-record `author` field. +- A new community-authored record, `social.coves.community.acceptance`, is a + strongRef (URI + CID) to an author's post. It is written **automatically by + the community's host** — no human in the loop — after admission checks pass. + For an open community and a non-banned author this happens in milliseconds. + Acceptance is machine attestation, not human approval. +- **Community surfaces render exclusively from admission state derived from + acceptance records.** A post *claiming* a community but lacking an acceptance + is never shown in that community, by lexicon-documented convention. (Same + trust shape as Bluesky threadgates.) +- Moderation removal = delete the acceptance record **and** write a + `social.coves.community.removal` record carrying an error code, in **one + atomic `com.atproto.repo.applyWrites` commit** (§5.5). Removal is a signed, + portable, auditable act that survives migration and forks. + +Policy on paper is unchanged — anyone can post in any public community unless +banned — but note honestly (§4.1): today's code does not yet enforce bans or +per-user rate limits, so the acceptance engine *introduces* those checks +rather than relocating them. + +**Verification boundary (do not overclaim).** "Author-signed" holds at the repo +layer: the record lives in a commit signed by the author's signing key. Our +ingestion path does not itself verify commit/MST proofs — Jetstream events +carry no proof material. The trust model is therefore: records are +author-attributed **as vouched for by a relay that verifies repo commits** (our +self-hosted relay does) **or by a direct, DID-resolved PDS fetch** (§5.4). +Consumers that ingest from unverified sources get repo-origin attribution, not +cryptographic proof. Document this in the lexicon descriptions; never describe +firehose events themselves as signed. + +### Why this preserves (and improves) the original goals + +- **Portability**: the community CAR carries the *curated index* — every + acceptance, removal, ban, rule, pin — instead of other people's prose. + Content liveness depends on author PDSs, exactly as it already does for + every comment thread. A snapshot dial (embedding content in acceptance + records) exists as a v2 option; v1 is pointer-only. +- **Forkability**: post URIs are `at://author-did/...` — they belong to no + community. A fork writes its own acceptance records pointing at the same + posts and the entire comment/vote graph stays attached. This requires + admission state to be **per-(community, post)**, not per-post — a post must + be able to hold independent admission decisions from multiple communities + (original + forks). §6.1 models exactly that; the post's `community` field is + only its *initial submission target*, and cross-community acceptance is the + privileged fork/import flow (§10.2). +- **Ownership across DIDs**: the community DID still owns everything + collective — curation, membership, bans, rules, governance outcomes. It + stops owning users' words, which was a liability (impersonation power, + hosting liability, deletion obligations), not an asset. + +--- + +## 3. Lexicon changes + +### 3.0 Evolution strategy — the lexicons are published + +The `social.coves.*` lexicons are published, so the atProto evolution rules +apply: non-optional fields cannot be removed, types cannot change, and +"larger structural changes require creating new Lexicons with different +NSIDs." Flipping a record's home repo is the largest structural change there +is — and keeping the old NSID would be actively dangerous, not just +rule-breaking: any consumer built against the published +`social.coves.community.post` derives **community = repo DID** for that +collection. Fed the same collection name from *author* repos, it would +silently index authors as communities. A new NSID makes stale consumers +ignore the new records entirely, which is the correct failure mode. + +Therefore: + +- The author-repo post record is a **new lexicon, + `social.coves.community.postv2`** — same namespace hierarchy as today + (deliberate owner choice over relocating to `social.coves.feed.*`; the + safety property comes from the NSID being *new*, not from where it sits). +- `social.coves.community.post` (the record) is **deprecated in place**: its + published schema is untouched except a description marking it deprecated + and pointing here. No new records are ever written to it; §10's migration + drains it. +- `social.coves.community.acceptance` and `social.coves.community.removal` + are brand-new NSIDs — no evolution constraints. +- The XRPC procedures/queries (`social.coves.community.post.create`, `.get`, + `.getStatus`, `.notify`) keep their NSIDs: the client contract evolves + additively (new optional output fields, new union members), which the + rules permit. A `community.post.*` endpoint family writing + `community.postv2` records is fine — endpoints are verbs, records are + nouns. + +### 3.1 `social.coves.community.postv2` (new — author repo) + +New file `internal/atproto/lexicon/social/coves/community/postv2.json` — the §3.0 +successor to `social.coves.community.post`, with these deltas from the +deprecated schema: + +- **Placement**: author's repo (joins `social.coves.feed.vote` and + `social.coves.community.comment`, which already live there). +- **No `author` field** — the repo DID *is* the author. Consumers MUST derive + authorship from the event DID. +- **Keep** `community` (required, `format: did`) — the initial submission + target. **Immutable across updates**: an update event that changes + `community` is invalid and MUST be ignored by consumers (retargeting a post + is a new post record). This prevents dangling admissions and matches the + existing update-immutability conventions in the consumer. +- **Keep** everything else: title, content, facets, embed union, langs, labels, + tags, crosspostOf/crosspostChain, createdAt, bridgedStats. +- `crosspostOf`/`crosspostChain` strongRefs now point at author-repo URIs — + stable across community forks (previously broken). + +### 3.2 `social.coves.community.acceptance` (new — community repo) + +New file `internal/atproto/lexicon/social/coves/community/acceptance.json`. +As specced in rev 1 (strongRef `subject` + `createdAt`, community implicit in +the repo), with two hardening changes from review: + +- **Record key: deterministic, not TID.** `key` is `any`, and the rkey is the + subject post's AT-URI transformed into rkey-safe form (strip `at://`, + replace `/` with `:` — e.g. + `did:plc:abc…:social.coves.community.postv2:3jzfcijpj2z2a`; well under the + 512-char rkey limit, and human-greppable). The record-key spec explicitly + blesses `any` for exactly this — "de-duplication and known-URI lookups" + via "a (transformed) AT URI" — and Bluesky's `threadgate` (rkey must equal + the subject post's rkey) is precedent for subject-derived keys. One post → + one acceptance rkey per community, forever. This makes the three + independent acceptance writers (sync fast path §4.3, firehose engine §5.6, + notify §7) **idempotent by construction** — concurrent attempts converge on + `putRecord` of the same rkey instead of allocating duplicate TIDs, and + re-acceptance after an edit is an update of the same record with a new + subject CID (references to the acceptance URI stay valid). +- Writes use `swapRecord`/`swapCommit` so a lost race is a detected conflict, + not a silent overwrite. + +The `subject` strongRef pins the accepted CID: if the author edits the post, +the CID no longer matches and the post is pending re-acceptance — clients and +AppViews MUST NOT auto-render the new CID under the old acceptance (§5.5). + +### 3.3 `social.coves.community.removal` (new — community repo) + +As specced in rev 1 (strongRef `subject`, required `code`, optional `reason`, +`createdAt`), plus: + +- `code` uses **`knownValues`, which is an open set by definition** — "values + are not limited to this set" — so new codes can ship without a lexicon + break. This is deliberate, per the draft Lexicon style guide's "enum sets + are closed … should almost always be avoided." Never convert it to `enum`. + Values are kebab-case (style-guide convention for fixed strings): + `rule-violation | spam | off-topic | illegal-content | author-banned | + moderator-discretion`. Add `maxLength: 64` so unknown codes are still + bounded. + +- Same deterministic-rkey scheme as acceptance (one removal record per + (community, post); re-removal updates it). +- **Removal is URI-scoped and terminal** (§5.5): it applies to the post URI, + not to the CID it happened to pin. The pinned CID is audit metadata. +- Written in the **same `applyWrites` commit** that deletes the acceptance, so + the firehose never carries a half-completed moderation action. + +Rejection at submission time (never accepted) writes **no record** — spam must +not bloat the community repo. Rejection state is AppView-local (§6.1) and +queryable via §3.4. + +### 3.4 XRPC surface + +- `social.coves.community.post.create` — kept as the client-facing procedure; + semantics per §4. **The service signature changes**: like comments' + `CreateComment`, it takes the caller's OAuth session + (`*oauth.ClientSessionData`) explicitly — no hidden context coupling — and + the aggregator path passes stored-token credentials instead + (`internal/core/posts/interfaces.go` and the handler change accordingly; + review confirmed the current interface takes no session at all). +- `social.coves.community.post.update` — **new scope, not a flip.** The route + is currently commented out (`internal/api/routes/post.go:54`) and the + service method doesn't exist. It ships in this change as an author-session + write with `swapRecord` conflict handling, because the edit → re-acceptance + lifecycle (§5.5) is core to the design and must be testable. +- `social.coves.community.post.get` — `postView` gains admission context + (status + acceptance URI for the viewed community); union gains + `#removedPost` (mirrors `#notFoundPost`/`#blockedPost`, carries the removal + `code`). URI normalization flips to author-DID-based. +- `social.coves.community.post.getStatus` — new query on the community host: + given post URI (+ community), returns + `accepted | pending | pending_reacceptance | rejected | removed` plus code + and decision time, read from the admissions table (§6.1). +- `social.coves.community.post.notify` — new procedure on the community host + (Beta, §7). + +--- + +## 4. Write path (`internal/core/posts/service.go`) + +### 4.1 What today's checks actually are (correcting rev 1) + +Review against the code: the current user flow enforces **community existence, +private-visibility block, embed/thumb validation, and aggregator +authorization + rate limits — nothing else**. The docstring's +"membership/ban validation" is aspirational; there is no ban lookup +(`ErrBanned` is marked Beta in `errors.go`) and no per-user rate limiting. + +Therefore `admitPost` (§5.6) is **extraction plus new policy**, not a +behavior-preserving refactor. New checks arriving with it, each with an +explicit error code and tests: ban lookup against indexed +`social.coves.moderation.ban` state, and per-author/per-community submission +rate limits (§8). The spec stops claiming otherwise. + +### 4.2 Flow + +1. Validate input, URI normalization, DID auth check — unchanged, order + preserved, fail-fast before any repo write. +2. **Blob handling flips to the author's PDS** under the author's session + (comments' `PDSClientFactory` + OAuth/DPoP pattern). The blob service + currently uploads with a community `BlobOwner`; it gains an author-repo + owner path. Embedded media is owned by the author's DID and PDS, as comment + embeds already are. +3. **Post record is written to the author's repo** via the author's session. + Aggregators (Kagi) write to their own repos via their stored OAuth tokens + (migration 025) — unchanged in shape. +4. **Local-community fast path**: this AppView hosts the community, so it *is* + the authoritative admission engine — it runs `admitPost` synchronously and + writes the acceptance (deterministic rkey, §3.2) via the existing + credential machinery (`EnsureFreshToken`). Client gets URI + CID back with + the post already accepted; UX identical to today. +5. **Remote-community path (Beta)**: the author's server does **not** run + admission checks — it has no authoritative view of a remote community's + bans, visibility, or quotas, and a stale or hostile home server must not be + able to fake either an admission or a rejection. Author-side + responsibilities are only: syntactic validation, authentication, and the + author-repo write. The post is `pending` until the community host decides + (§7); the client learns the outcome via optimistic self-view + `getStatus`. + +Failure mode: author-repo write succeeds, acceptance write fails → post stays +`pending`; the firehose engine (§5.6) retries idempotently (same rkey). +Degraded latency, not data loss. Never roll back the author's record. + +`post.delete` likewise flips to an author-session delete. + +--- + +## 5. Ingestion (`internal/atproto/jetstream/`) + +### 5.1 `consumerWantedCollections` (`feeds.go:56`) + +```go +ConsumerPosts: { + "social.coves.community.postv2", + "social.coves.community.acceptance", + "social.coves.community.removal", +}, +``` + +The deprecated `social.coves.community.post` collection is dropped from the +subscription — after §10's drain, no live events exist for it, and its delete +events during migration are irrelevant because `posts` is truncated and +re-indexed anyway. + +`cmd/contract-manifest` then fails CI until each new collection has a +`//coves:ingestion-contract` marker in `tests/e2e/` — the mechanism that +forces §9's pipeline tests to exist. + +### 5.2 Ordering: the per-record rev gate is not enough + +Acceptance and removal are **different record URIs about the same subject**, +so the existing per-record rev gate cannot order their combined effect: a +redriven stale acceptance could resurrect a removed post; a delayed acceptance +delete could flip `removed` back to `pending`. + +Fix: admission state transitions are gated by a **subject-scoped watermark** — +`last_community_rev` on the admissions row (§6.1). Any event from the +community's repo about subject S applies only if its commit rev is newer than +the row's watermark; within a single rev (the atomic delete-acceptance + +create-removal commit, §3.3/§5.5), **removal wins**. Author-repo events (post +create/update/delete) keep the existing per-record rev gate. All handlers +remain idempotent upserts; replay is a no-op. + +### 5.3 Post events (author repos) + +- Author DID = `event.Did`. Community DID = the record's `community` field; + unknown community → dead-letter (redrive resolves the profile race). +- **Unknown authors — the FK must go.** `posts.author_did` currently has a + hard FK to `users` with `ON DELETE CASCADE` (migration 011), and the + consumer only bootstraps missing authors from trusted bridge PDSs — the test + architecture states federated authors cannot currently be indexed. That + contradicts open federated posting. Changes: drop the FK to a soft + reference (migration 034), hydrate unknown author profiles opportunistically + via SSRF-safe identity resolution (extending the existing + `identityResolver` path beyond bridge trust), and **stop cascading** — a + deleted profile row must not silently erase indexed posts; author-requested + deletion is an explicit tombstone (existing comments pattern, migration 021). +- Index/refresh the post row; admission state lives in §6.1, initial status + `pending` for the target community. +- Update events: §5.5. Delete events: tombstone the post; the community host + observes the tombstone and deletes its acceptance (removal record optional — + author deletion is not moderation). + +### 5.4 Acceptance/removal events (community repos) — and convergence + +- Repo DID must be an indexed community; else dead-letter. +- **Acceptance-before-post does not converge by redrive alone** — bounded + dead-letter retries cannot manufacture a post event that a relay-coverage + gap will never deliver. On acceptance whose subject post is unindexed: + resolve the subject author's DID → PDS, fetch the record directly + (`com.atproto.repo.getRecord`), **verify the returned CID equals the pinned + CID**, apply strict SSRF protections and size/time caps, and index it. The + DLQ remains the backstop for transient failures only. Notify (§7) is a + latency optimization; direct fetch is what makes firehose-only ingestion + actually converge. State plainly: without either, convergence requires full + relay coverage. +- Acceptance apply (subject to §5.2 watermark): pinned CID == indexed post + CID → `accepted`; mismatch → `pending_reacceptance` (stale acceptance for a + since-edited post; the engine re-emits). +- Acceptance delete: leaves `accepted`; goes to `removed` if the same-rev + removal is present (it will be, §3.3), else `pending`. +- Removal apply: `removed` + code. A removal without prior acceptance is + valid (pre-emptive) and indexes normally. + +### 5.5 Edits, re-acceptance, and removal terminality + +- Post update where the new CID ≠ accepted CID → `pending_reacceptance`; the + engine re-runs `admitPost` on the new content and either updates the + acceptance (same rkey, new pinned CID) or removes with code. Edited content + is never auto-rendered under the old acceptance. +- **Exception — `bridgedStats`-only updates.** Bridges refresh records + frequently *specifically* to update origin-platform vote counts; treating + each refresh as an edit would strobe accepted bridge posts out of feeds and + spam the community repo with re-acceptance commits. Rule: if the record diff + touches **only** `bridgedStats` and the author passes the bridge-trust gate, + repin synchronously (acceptance update, no status change, no feed removal). + Any diff touching title/content/facets/embed/labels/tags/community requires + full re-admission. (v2 alternative if this proves noisy: move bridged + aggregates to a separate author-owned stats record so content CIDs stop + churning at all.) +- **Removal is terminal across author edits.** `removed` is exited only by an + explicit moderator restore (atomic: delete removal + write fresh + acceptance). An author edit while removed updates audit metadata only — + otherwise editing would launder a removed post back through auto-acceptance. +- Bridge-trust re-keying: the `BridgeTrust` gate currently trusts *community* + repos' PDS provenance for `bridgedStats`; it re-keys to the **author** + (bridge account) repo's PDS. + +### 5.6 Acceptance engine (new component) + +The single decision point: inputs are pending/pending-reacceptance admissions +for communities this AppView hosts (from the fast path, the firehose consumer, +or notify). It runs `admitPost` — the extracted §4.1 checks plus the new +ban/rate-limit policy — then writes/updates the acceptance, or (rejection) +records the decision in the admissions table, or (re-acceptance failure / +moderation) performs the atomic acceptance-delete + removal. It is the only +writer of community-repo records in the post system, and every write is +idempotent via deterministic rkeys + swap semantics. + +--- + +## 6. Storage & read paths + +### 6.1 Migration `034_author_owned_posts.sql` + +**Posts stay pure content; admission state is relational.** A post must be +able to carry independent decisions from multiple communities (forks, §2), a +`rejected` state must actually exist somewhere queryable (rev 1 promised it in +`getStatus` but stored it nowhere), and decisions need audit metadata. + +New table `community_post_admissions`: + +- PK `(community_did, post_uri)` +- `status TEXT NOT NULL` — + `pending | accepted | pending_reacceptance | rejected | removed` +- `acceptance_uri`, `acceptance_rkey`, `accepted_cid` (NULL unless accepted) +- `decision_code TEXT`, `decision_at TIMESTAMPTZ` — rejection/removal codes; + rejections are AppView-local (never a community-repo record) +- `evaluated_cid TEXT` — the exact CID the last decision judged +- `redrivable BOOLEAN NOT NULL DEFAULT true` — policy rejections are + `false` (terminal; not retried by DLQ redrive); transient evaluation + failures stay `true` +- `last_community_rev TEXT` — the §5.2 subject-scoped watermark +- Partial index on `status = 'accepted'` for feed queries + +`posts` changes: drop the `users` FK + CASCADE (§5.3), drop the in-record +author column's trust role (author = repo DID), keep content columns. +Pre-production data: truncate and re-materialize (§9). + +### 6.2 Read paths — centralized visibility, full inventory + +Review found rev 1's list incomplete (e.g. `getComments` hydrates its post via +raw `GetByURI` and already serves deleted-post content today). Piecemeal +predicates will miss a surface; therefore: + +- **One admission-aware visibility predicate** (SQL view or shared query + helper joining `community_post_admissions`), used by *every* posts read + path. No handler queries `posts` directly for display. +- Inventory to convert and test: community feeds (`feed.getCommunity`, + `getAll`, `getDiscover`, `getTimeline`, `communityFeeds`), `post.get`, + `getComments`' post hydration, actor surfaces (`actor.getPosts`, comment + community filters), search, embed hydration of quoted posts, community post + counts and user statistics (counts must not include non-accepted rows), and + moderation views (which deliberately *do* see pending/removed). +- Author self-view: authors see their own posts with per-community status; + other viewers see accepted only; removed renders as `#removedPost` + code; + pending renders to non-authors as `#notFoundPost`. +- T1/T2 tests must prove pending/removed content is unreachable through the + *alternate* endpoints (comments, search, counts), not just absent from feeds. + +--- + +## 7. Remote communities (Beta — sequenced after local flip) + +**Plain-language scope note.** This section is ONLY about the cross-server +case — a user whose home server is instance A posting into a community hosted +by instance B. It has nothing to do with private communities. The problem it +solves: after the author's post lands in their own repo on A, *B has to find +out it exists* before B's acceptance engine can evaluate it. Firehose +delivery gets it there eventually — if B's relay crawls A's PDS — but that is +a coverage-and-latency bet. `post.notify` is A tapping B on the shoulder: +"post at `` targets your community, go evaluate it now," turning +post-to-accepted from "whenever the relay delivers" into one round trip. +`getStatus` is the reverse direction: the author's client asking B "did you +accept it, and if not, why" (B's rejections are AppView-local, §3.3, so +there's no record to read — you have to ask). Service auth is how B knows the +notify/getStatus caller genuinely is that author, without the author having +any account on B. For same-server posts none of this machinery runs — §4.4's +synchronous fast path already did everything. + +federation-prd's service-auth material is explicitly hypothetical, and the +current middleware verifies service JWTs only for registered aggregators, +without `lxm` enforcement. This section is therefore a specification to build, +not machinery to reuse: + +1. **Discovery**: the community DID document gains a concrete service entry — + id `#coves_host`, type `CovesCommunityHost`, endpoint = the hosting + AppView's public URL — written at community creation and rotated on + migration. Resolvers validate scheme/host (https, no private ranges). +2. **Auth**: `post.notify` and `post.getStatus` accept atProto service-auth + JWTs: `iss` = the *author's* DID, `aud` = the community host's service DID, + `lxm` = the exact method NSID, short expiry, `jti` replay cache. Endpoint- + specific middleware — not the aggregator-global path — enforces all four. +3. **Flow**: author's server writes the post to the author's repo, then + service-proxies `post.notify` to the community host. The host fetches the + record from the author's PDS (CID-verified, SSRF-safe — same fetch path as + §5.4), runs the acceptance engine, writes the acceptance; its firehose + carries the acceptance to every other AppView. +4. Without notify, §5.4's direct fetch on acceptance-events plus relay + coverage still converges; notify removes the latency. + +Client UX: optimistic self-view immediately; `getStatus` polling for the +accepted transition. + +--- + +## 8. Abuse & resource limits + +Anyone can write unlimited posts naming any community; nothing stops the +*records* — the design absorbs them at the admission layer: + +- Per-author, per-community, and per-origin-PDS submission quotas in the + acceptance engine (new policy, §4.1), with `rejected` + + `rate-limit-exceeded` decision codes, `redrivable = false`. +- Dedupe identical submissions by (author, community, content CID). +- Debounce edit re-evaluation per post (a rapid edit storm collapses to the + latest CID). +- Retention caps on `pending`/`rejected` admission rows for never-accepted + posts. +- Notify endpoint: per-caller and per-PDS quotas on top of service-auth. +- All outbound fetches (identity bootstrap §5.3, record fetch §5.4/§7) behind + SSRF guards, response-size caps, and timeouts. + +--- + +## 9. Testing (per `docs/TEST_ARCHITECTURE.md`) + +- **T0**: `admitPost` matrix (visibility, ban — new, rate limits — new, + aggregator authz); §5.5 state machine including bridgedStats-only repin, + removal terminality, edit-while-removed; deterministic rkey derivation; + lexicon fixtures (post loses `author`; acceptance/removal fixtures valid + + invalid; community-immutability rejection). +- **T1**: admissions-table transitions under the §5.2 watermark (stale-rev + acceptance after removal is a no-op; same-rev removal wins); + acceptance-before-post → direct-fetch path (CID mismatch rejected); + rejection rows non-redrivable; idempotent replay of fast-path + firehose + double-delivery. +- **T2**: ingestion contracts for both new collections. Full loops: author + write → pending → accepted → in feed; banned author never accepted; edit → + pending_reacceptance → re-accept; moderator removal (atomic commit) → feed + drop + `#removedPost`; author delete → tombstone + acceptance cleanup; + pending/removed unreachable via comments/search/counts. +- Anti-flake: subscribe-before-write cursor pattern; no sleeps (test-audit). + +## 10. Rollout (hard cutover — pre-production) + +### 10.1 Order + +One branch, one `make ci`, one merge: + +1. Lexicons (§3) + fixtures + migration 034. +2. `admitPost` extraction **plus** the new ban/rate-limit checks with tests + (called out as new policy, not refactor). +3. Ingestion (§5): admissions state machine, watermark, direct-fetch + convergence, consumers + contracts. +4. Write path (§4: session-explicit signature, author-repo writes, author-repo + blobs, `post.update` as new scope) + read path (§6.2 centralized + predicate). +5. Prod cutover: **stop writers** (brief maintenance window — mixed old/new + writers must not overlap), then a **resumable, idempotent migration script + with a per-post ledger**: for each community-repo post, write the record + into the author's repo as `social.coves.community.postv2` (dropping the `author` + field; authors are overwhelmingly the Kagi aggregator, + whose session we hold; human authors are on our PDS), write the acceptance, + **verify both records and the pinned CID, checkpoint the ledger — and only + then** delete the community-repo record. Authors whose credentials cannot + be restored are logged and their posts re-materialized under an explicit + fallback decision, not silently dropped. Truncate + re-index `posts`. +6. Delete: community-credential post writes, `author`-field handling, old + fixtures. `federation-prd.md` gets a superseded-by header. + +### 10.2 Client coordination + +`coves-mobile` / `coves-frontend`: post URI authority flips to author DIDs; +anything parsing `at://` URIs or deep-linking by community DID changes in the +same release window. Pending/removed status UI ships after cutover. The +fork/import flow (a community accepting a post whose `community` field names a +different DID) is deliberately **not** built now — the data model supports it +(§6.1), the privileged flow that exercises it is future scope. + +## 11. Open questions (decided-by-default, flag to revisit) + +1. **Snapshot dial** (v2): embed content in acceptance records for + self-contained CARs — at the cost of re-hosting user content. Default: + pointer-only. +2. **Comment admission**: comments render un-gated on accepted posts. + Default: out of scope. +3. **Separate bridged-stats record** (v2): if bridgedStats-only repins (§5.5) + still generate too much community-repo churn, move mutable aggregates out + of the post record entirely. +4. **Private communities** (Beta): acceptance records leak subject URIs on the + public firehose; private-community design must address opaque/hashed + subjects. Out of scope for the flip. diff --git a/loop_state.md b/loop_state.md new file mode 100644 index 0000000..6e38eb2 --- /dev/null +++ b/loop_state.md @@ -0,0 +1,39 @@ +# Author-owned posts build-loop state + +Protocol: `plan.md` §Loop protocol — one task per iteration: +analyze (parent) → **/tdd** (Opus 5 phase agents — never downgrade) → +**/second-opinion** (base: `feat/author-owned-posts`) → **/fix-pr** → +`make ci` → `--no-ff` merge into `feat/author-owned-posts` → delete task +branch/worktree → update this file → next task. +Spec authority: `docs/PRD_AUTHOR_OWNED_POSTS.md` (rev 2.1). +`main` is touched once, at the end, via /merge-to-main after task 8. +Stop when every task is done, or on any `blocked:` row. + +## Task table + +| # | Task | Phase | Status | Merge commit | Notes | +|---|------|-------|--------|--------------|-------| +| 1 | Lexicons: postv2 + acceptance + removal, deprecation note, fixtures, T0 validation | A | pending | | | +| 2 | Migration 034: community_post_admissions + posts FK drop; admissions repo + T1 transition/watermark matrix | A | pending | | | +| 3 | admitPost extraction + NEW policy (bans, rate limits, dedupe) wired into existing write path | A | pending | | | +| 4 | Acceptance engine: deterministic rkey, swap-safe acceptance writer, atomic applyWrites removal, repin/terminality rules | B | pending | | | +| 5 | Ingestion: postv2/acceptance/removal consumers, watermark gating, direct-fetch convergence, WantedCollections + 3 e2e contracts | B | pending | | parent may split 5a/5b at brief time; WantedCollections + contracts same merge | +| 6 | Write path flip: author-repo postv2 via session, author-PDS blobs, sync fast-path accept, post.delete flip, post.update NEW | C | pending | | review MUST include pragma:security — verify it fired | +| 7 | Read path: centralized visibility predicate, full surface inventory, #removedPost, getStatus, alternate-endpoint invisibility T2s | C | pending | | | +| 8 | Cutover: re-materialization script (ledger, verify-before-delete, hermetic test), old-path removal, docs, tracker cleanup; panel on whole branch; /merge-to-main | D | pending | | prod script run is MANUAL, outside loop | + +## Cross-iteration notes + +- (2026-08-07) Loop scaffolded. Owner decisions locked in PRD rev 2.1: + new NSID `social.coves.community.postv2` (not feed.post — hierarchy kept + deliberately); `post.notify` + service auth deferred to Beta follow-up; + lexicons are PUBLISHED — never edit `community.post`'s schema, deprecation + note only. +- /tdd model check (2026-08-07): skill default is opus (= Opus 5 here); + `sonnet`/`haiku` exist only as explicit downgrade modifiers — never pass + them. +- Known adjacent defect (pre-existing, do not fix inline): community blocks + indexed but never enforced (issue 2026-07-29) — task 7's inventory will + touch the read paths where this surfaces; keep it filed, don't scope-creep. +- Kagi/aggregator posts are the bulk of prod data; task 8's script leans on + the aggregator's stored OAuth session (migration 025) for re-authoring. diff --git a/plan.md b/plan.md new file mode 100644 index 0000000..715f168 --- /dev/null +++ b/plan.md @@ -0,0 +1,204 @@ +# Author-owned posts — build-loop plan + +**Spec: `docs/PRD_AUTHOR_OWNED_POSTS.md` (rev 2.1). This file is the +implementation partition + loop protocol; the PRD is the design authority. +Where they disagree, the PRD wins — fix the PRD first, then the code.** + +Tracker: `loop_state.md` (living state, one row per task). + +## Branch strategy + +One **integration branch**, `feat/author-owned-posts`, cut from `main` at +loop start. Each task runs on its own short-lived branch cut from the +*current integration head* (the /tdd worktree provides this isolation), +and folds back in at iteration end: + +``` +main ──────────────────────────────────────────────▶ (single /merge-to-main + └─ feat/author-owned-posts ──M1───M2───M3─── … ──▶ after task 8's panel) + └ tdd/task-1 ──╯ │ │ + └ tdd/task-2 ─╯ │ + └ tdd/task-3 ─╯ +``` + +- Merges into the integration branch are **`--no-ff`** — one merge commit + per task keeps the task boundary in history (the audit trail the + stacked-branch alternative would have provided, without its rebase + ceremony — the tasks are strict sequential dependencies, so true + stacking buys nothing here). +- Task branches + worktrees are deleted after their merge (the "reset"). +- `main` is touched exactly once, at the very end, via `/merge-to-main` + after task 8's final review panel. `make ci` is the merge gate at BOTH + levels: before every integration-branch merge, and again at the final + merge to main. + +## Loop protocol (one iteration per task) + +1. **Analyze** (parent, persistent context): read `loop_state.md`, pick the + first non-done task, write a **self-contained brief** — scope, files, + PRD sections, relevant cross-iteration notes, exit criteria, and the + task's acceptance behavior (the outer BDD test /tdd will frame). +2. **`/tdd `** — the whole implementation happens inside the tdd + skill: parent as CONDUCTOR, outer acceptance test, inner red/green/ + refactor cycles, RED author and GREEN implementer as **separate + persistent Opus 5 agents** (the skill's default model is opus — NEVER + pass the `sonnet`/`haiku` modifiers). Worktree on a task branch cut + from the integration head. +3. **`/second-opinion`** on the completed task, scoped to the task diff — + tell it the base explicitly: `git diff feat/author-owned-posts...HEAD`. + pragma:security fires automatically on trust-boundary diffs; task 6 + (auth/credential surface) must include it — verify it fired, don't + assume. +4. **`/fix-pr`** with the synthesized review report pasted in. Re-run the + tdd suite after fixes land (fix-pr's own go vet/build check is not the + gate). +5. **Verify**: full `make ci` in the task worktree. Green or it doesn't + merge — red means fix now, or mark the task `blocked: ` in + loop_state and stop the loop for user input. +6. **Merge** the task branch into `feat/author-owned-posts` (`--no-ff`, + merge commit titled `task N: `), delete the task branch and + worktree. +7. **Update `loop_state.md`** (status, merge commit, notes — surprises and + decisions go to Cross-iteration notes) and schedule the next iteration. + +Statuses: `pending → in-tdd → review → done` (or `blocked: `). +Stop when every task is done or on any block. + +Standing rules (carried from the test-refactor loop): +- A skip is a failure; missing infra is a `t.Fatal` naming the target. +- Subscribe-before-write cursor pattern for every firehose wait; no sleeps. +- Test handles: suite canonical helpers only (`uniqueTestID`/`uniqueAccount`). +- Defects found in *existing* code that are out of task scope: /file-issue, + don't fix inline, note in loop_state. +- Briefs to any agent are self-contained (file:line, failure scenario, + expected behavior) — no agent reads this file's history to reconstruct + context. + +## Task partition (each row ≈ one PR-sized /tdd run) + +Sizing rationale: every task leaves `make ci` green. The contract-manifest +gate forces consumers and their e2e contracts to land together (task 5); the +read-path filter (task 7) lands *after* the write flip (task 6) so +status-agnostic reads keep serving both old and new posts during the middle +of the loop. + +### Phase A — foundations (no behavior change) + +**1. Lexicons + fixtures** +PRD §3. New `internal/atproto/lexicon/social/coves/community/postv2.json` +(no `author`, `community` immutable-on-update documented), +`community/acceptance.json` (key `any`, deterministic-rkey doc, strongRef +subject), `community/removal.json` (open `knownValues` code + maxLength 64). +Deprecation note in `community/post.json` description ONLY (schema +untouched — it's published). Fixture tree: `tests/lexicon-test-data/postv2/` +valid+invalid, `acceptance/`, `removal/`; extend the record-lexicon +validation T0 tests. Acceptance behavior for the outer test: the three new +lexicons validate their valid fixtures and reject each invalid fixture with +the expected error. Exit: T0 green; no consumer references yet. + +**2. Migration 034 + admissions repository** +PRD §6.1, §5.3. `034_author_owned_posts.sql`: `community_post_admissions` +(PK (community_did, post_uri), status, acceptance_uri/rkey, accepted_cid, +decision_code, decision_at, evaluated_cid, redrivable, last_community_rev, +partial index on accepted); drop `posts` FK to users + CASCADE (soft ref). +`internal/db/postgres` admissions repo: upserts, status transitions, +watermark compare-and-set. Acceptance behavior: the T1 transition matrix — +stale-rev events are no-ops, same-rev removal wins, every legal/illegal +status transition. Exit: T1 green. + +**3. `admitPost` extraction + real admission policy** +PRD §4.1, §8. Extract CreatePost's checks into a shared `admitPost` +(community exists, visibility, aggregator authz) and ADD the new policy: +ban lookup against indexed ban state, per-author/per-community submission +rate limits, dedupe by (author, community, content CID) — each with typed +error → decision code. Wire into the EXISTING write-forward path (bans +start being enforced now; that's intended policy, call it out in the merge +commit). Acceptance behavior: a banned author's create is refused with the +ban code end-to-end on the current path. Exit: T0 matrix covers every +branch; T1 for ban/rate lookups; e2e still green. + +### Phase B — the new machinery + +**4. Acceptance engine + community-repo writers** +PRD §3.2, §3.3, §5.5, §5.6. Deterministic rkey helper (subject AT-URI → +rkey-safe transform) with T0 round-trip tests; acceptance writer +(putRecord + swapRecord conflict path); atomic acceptance-delete + removal +via `com.atproto.repo.applyWrites`; engine core: consume a +pending/pending_reacceptance admission → admitPost → write/update +acceptance | record rejection (AppView-local, redrivable=false) | atomic +remove. Removal terminality + bridgedStats-only repin decision function +(pure, T0). Acceptance behavior: engine double-fired on the same subject +converges to exactly one acceptance record (idempotence), proven against +the real PDS container at T1. Exit: T0/T1 green; nothing triggers the +engine in prod paths yet. + +**5. Ingestion: postv2 + acceptance + removal consumers, contracts** +PRD §5 entire. postv2 handler in `PostEventConsumer` (author = event.Did, +community from record + immutability enforcement, unknown-author soft +handling + opportunistic SSRF-safe identity hydration, pending admission +row, edit → CID compare → pending_reacceptance, delete → tombstone); +acceptance/removal handlers (community-repo authority check, §5.2 watermark +gating, acceptance-before-post → CID-verified direct PDS fetch with +SSRF/size/time caps, DLQ backstop); engine triggered for hosted +communities; BridgeTrust re-keyed to author repo. +`consumerWantedCollections` gains the three collections; THREE +`//coves:ingestion-contract` e2e contracts. Acceptance behavior (the outer +test IS the primary contract): postv2 written to an author repo on the +hermetic PDS → firehose → pending → auto-accept → accepted; plus banned +author never accepted; edit → re-accept; removal → atomic commit observed. +Old `community.post` consumer path stays live in parallel. This is the +hardest task — the parent may split it 5a (consumers) / 5b (contracts) at +brief-writing time, but WantedCollections + contracts must land in the +same merge. Exit: full `make ci` green. + +### Phase C — the flip + +**6. Write path flip** *(review must include pragma:security)* +PRD §4. CreatePost: session-explicit signature +(`*oauth.ClientSessionData`, comments' `PDSClientFactory` pattern), write +postv2 to author repo, blobs/thumbnails to author PDS (blob service gains +author BlobOwner path), aggregator stored-token path, sync fast-path +acceptance via engine, no rollback of author records on acceptance +failure. `post.delete` → author-session delete. `post.update` NEW endpoint +(route, handler, service, swapRecord conflict handling). Update e2e flows +that create posts. Acceptance behavior: a user's create lands in THEIR +repo, is accepted synchronously, and round-trips through the feed; an +acceptance-write failure leaves the author record intact and pending. +Exit: full `make ci`. + +**7. Read path: centralized visibility** +PRD §6.2, §3.4. One admission-aware predicate (view or shared query +helper); convert the inventory: community feeds ×4 + communityFeeds, +post.get (+ `#removedPost` union + admission context in postView), +getComments post hydration, actor surfaces, search, embed hydration, +counts/stats. `getStatus` query endpoint. Author self-view semantics. +Acceptance behavior: pending/removed posts are unreachable through EVERY +alternate endpoint (comments, search, counts) for non-authors, while the +author still sees status. Exit: full `make ci`. + +### Phase D — cutover + +**8. Drain, cleanup, cutover tooling** *(final panel reviews the whole branch)* +PRD §10. Re-materialization script (`cmd/` tool): per-post ledger, +resumable/idempotent, write postv2 to author repo → acceptance → verify +both + pinned CID → checkpoint → delete old community-repo record; tested +hermetically against the e2e stack with seeded old-style data. Remove: old +`community.post` from WantedCollections + its consumer branch + +community-credential post writes + `author`-field handling; old fixtures. +`federation-prd.md` superseded-by header. Docs pass over the PRD (mark +implemented, log divergences). Delete `plan.md` + `loop_state.md` before +the final merge (build-loop trackers, not suite artifacts — precedent: +`512e00e`). Acceptance behavior: the script, run twice against a seeded +hermetic stack, converges with a complete ledger and zero old-style +records. Exit: full `make ci`; task's /second-opinion runs on the WHOLE +branch diff vs main; then `/merge-to-main`. Prod execution of the script +is MANUAL, outside the loop, coordinated with coves-mobile/coves-frontend +URI-parsing updates (PRD §10.2). + +### Deferred (NOT loop tasks) + +- **Beta remote path** (PRD §7): `post.notify`, service-auth middleware + (`lxm`/`aud`/replay), DID service entry, local-vs-remote detection. + Explicitly deferred per owner decision 2026-08-07. +- Private-community admission leakage (PRD §11.4), snapshot dial (§11.1), + separate bridged-stats record (§11.3). -- 2.51.2