diff --git a/openspec/changes/release-witness/.openspec.yaml b/openspec/changes/release-witness/.openspec.yaml new file mode 100644 index 0000000..9567240 --- /dev/null +++ b/openspec/changes/release-witness/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-08-18 diff --git a/openspec/changes/release-witness/design.md b/openspec/changes/release-witness/design.md new file mode 100644 index 0000000..b6d449c --- /dev/null +++ b/openspec/changes/release-witness/design.md @@ -0,0 +1,92 @@ +# Design: Release Witness + +## Context + +atproto repos are signed Merkle Search Trees: any record can be proven present in a repo with a CAR slice (record block + MST path + signed commit) verified against the account's DID document — the primitive `com.atproto.sync.getRecord` already serves. dist.town's release records are content-addressed (artifact digests live on the descriptor), so the chain *file bytes → digest → release record → commit signature → publisher DID* is already verifiable end to end without trusting dist.town. + +What the substrate cannot provide alone: trusted time, freshness ("still un-yanked, still the current pointer target"), and detection of publisher key compromise. The release lexicon's existing prose norm — "any update to an existing release record is treated by indexers as a probable-compromise signal" — has no signed artifact behind it. This change gives it one, and packages the whole chain for the cheapest imaginable verifier: a firmware self-updating device below ESP32 class (Cortex-M0 / 8-bit, tens of KB of RAM, no TLS stack, no reliable clock). + +Constraints: the moderation spec's firewall ("publisher state never lives in labels") and its anticipation of attestation services as first-class labelers; the service-identity spec's one-identity rule; publisher-facing records (release, pointer, status) must not change. + +## Goals / Non-Goals + +**Goals:** +- Signed, timestamped, publicly auditable observations of every release and of anomalous mutations, as records in the witness's own repo. +- A device-facing resolve endpoint whose responses verify with SHA-256 + a handful of ECDSA operations, with no parsing, no TLS, and no clock on the device. +- Device-side policy primitives that convert publisher key compromise from instant-and-silent to slow-and-public. +- A witness role that any operator can fill: the lexicon is the whole interop contract. + +**Non-Goals:** +- Preventing publisher key compromise (converted to detectable, not prevented). +- Build provenance / SLSA-style attestation of how artifacts were produced (possible future interop). +- Full certificate-transparency guarantees (commit-head gossip between witnesses is future hardening; repo structure supports it). +- Pointer retarget history (deferred; legitimate and chatty). +- Byte-level artifact attestation ("I fetched URL U and got digest D") — devices hash bytes themselves; per-location health already exists in `locationView`. + +## Decisions + +### D1: Observations are records in the witness's repo, not labels +Labels are a loose stream — a misbehaving witness could show different subscribers different sets, and label signatures would be a second verification code path on devices. Records in an MST give: one signed root per commit (equivocation within a commit is impossible, forks are detectable by any firehose mirror), the same verifier code path devices already need for the publisher proof, and accountability symmetry — mutation of witness records is itself an anomaly observable by anyone, under the same norms the witness enforces. Labels remain as a derived distribution rail (D6). Alternative considered: labels-only (rejected — weaker log structure, second device code path); both-as-peers (rejected — two sources of truth). + +### D2: The subject is a strongRef; observations embed nothing +`subject: {uri, cid}` pins the exact bytes observed. The device already holds the release record bytes from the publisher proof; hashing them once yields the CID at which the two proof chains meet. No snapshot duplication, nothing to drift. + +### D3: Neutral event vocabulary; `priorCid` yes, chain links no +Events `appeared` / `mutated` / `deleted` (open vocabulary) are measurements, not judgments — modified is not compromised (innocent cases exist: a digest typo fixed seconds after publish, migration replays). Severity interpretation is the consumer's layer. + +`mutated`/`deleted` carry `priorCid` — the previously observed subject CID. This is a fact about the *subject*, and the state needed to write it (last-seen CID per subject) is state the mutation detector must hold anyway. + +Rejected: a `prior` link to the previous observation record (per-subject chain). It couples every write to the identity of the previous write, forking silently under concurrent workers or at-least-once replay; broken topology can never be repaired in place under the witness's own append-only norms; and it buys nothing — the repo is already a hash-linked signed structure, and per-subject history is a single indexed scan in TID order. + +### D4: `observedAt` is the semantic clock +`observedAt` is firehose receipt time, meaning "this record existed *no later than* this moment." It is never backfilled from publisher-supplied data (record contents, commit claims). The observation's TID is storage order only and may skew from `observedAt` under batching. `repoRev` (the publisher's commit rev at observation) is the third clock, tying observations to points in the publisher's commit sequence for forensics. + +### D5: Universal coverage +Every `town.dist.release` on the firehose is observed, not just opted-in projects. Universality is what makes the log trustworthy (no registration step, no gap an attacker can hide in), and release volume is low. Mutation/deletion detection covers release and status records — the two types whose existing norms declare them immutable/append-only. + +### D6: Labels are derived from records, named as measurements +The existing labeler emits e.g. `release-modified` derived from `mutated` observations, reusing moderation rails that already reach every subscribed surface. The record is the verifiable artifact behind the label. Vocabulary stays visibly distinct from judgment labels (`malware`, `spam`): mechanical observations may be auto-acted on; judgments are advisory. This satisfies the moderation spec's firewall — observations are service measurements about publisher records, not publisher state. + +### D7: The resolve endpoint is a transcoder, not an oracle +The trust question for lifting work off the device: how much can the server do while remaining unable to lie? Answer: all the parsing, none of the trust. The endpoint decodes CAR/DAG-CBOR, walks the MST, assembles the PLC operation chains, and emits a flat, fixed-offset `application/octet-stream` blob of hash preimages: the device hashes exactly the bytes it is handed, checks each digest appears at the stated offset of the next preimage, and finishes with ECDSA verifies against pinned identities. A malicious server can produce a blob that fails (DoS) but never a passing blob for tampered content. The endpoint lives in the lexicon as a query with binary output, beside `resolveRelease`. + +Freshness is nonce-based: the device sends a random nonce; the witness countersigns (release, current lifecycle state, current pointer target, nonce). Clock-free, replay-proof. Mutable/absence facts ("not yanked *now*") live only here — a label or record can never say them, because a replay attacker shows the old positive and withholds the negation. + +The nonce-response signature uses a dedicated verification method declared in the witness's DID document, not the repo signing key raw — domain separation, one PLC op chain delivers both keys. + +### D8: Device policy primitives +- **Maturity window**: accept only releases with `observedAt` ≥ N hours old — device-chosen policy, not witness policy (a toy fleet picks 72h, a CLI picks zero). This is the defense against the attack the witness cannot detect: a *fresh* malicious release signed with a stolen publisher key looks legitimate. The window forces the attacker to survive N hours in public, against the yank button. +- **Anti-rollback**: device stores `lastSeenTid`; TID rkeys are strings whose order is creation order, so downgrade protection is a string compare. +- **Pinning**: publisher DID + project + pointer name at manufacture. Channels are pointers; per-fleet pointers (e.g. `fleet-mk3`) give post-manufacture fleet steering with zero new record types. +- **Re-release norm**: legitimate rollback past anti-rollback is a *new* release record referencing the old artifact digests (records are tiny; artifacts are content-addressed). Documented publisher norm — without it, a fleet bricks itself on its first bad release. +- **Key resolution**: publisher and witness are did:plc; devices verify rotation via the self-certifying PLC op chain, delivered incrementally in the blob (device caches last-verified key + op index). + +### D9: Defined, not owned +`town.dist.witness.observation` is the entire operator contract: any party watching the firehose writes the same collection into its own repo under its own did:plc. Device makers pin one or several witness DIDs and may require quorum on `subject.cid`. dist.town runs the reference witness and defines the vocabulary; nothing in the protocol privileges it — the same posture the moderation spec takes for labelers and the storage union takes for hosts. + +### D10: Blocked on the service-account migration +Witness records need a repo; decade-scale device pinning needs self-certifying key rotation. did:web has neither. The separate `service-account-migration` change gives dist.town a did:plc account (handle `dist.town`) on its own single-account PDS, re-homing labeler/report/admin identities. This change consumes that identity. + +## Risks / Trade-offs + +- [Fresh malicious release passes the witness — it looks legitimate] → Maturity window + universal public visibility make the attack slow and loud; yank (status record) plus nonce-freshness cuts it off fleet-wide. Residual: consumers that opt out of maturity windows get today's security, not less. +- [Availability dependence: devices need a reachable witness endpoint for freshness] → Integrity never depends on availability (proofs verify offline); quorum/multi-witness pinning removes the single operator; stalled updates are the failure mode, not compromise. +- [Witness equivocation across commits / withheld observations] → Fork detection by firehose mirrors; universal coverage means gaps are observable; commit-head gossip between witnesses is deferred hardening. +- [PLC directory dependency] → The op chain is self-certifying and verified on-device from bytes the transcoder supplies; the directory is discovery, not a trust root. +- [secp256k1 publishers are expensive for embedded verifiers (no hardware support in cheap secure elements)] → Documented guidance: publishers targeting device consumers use P-256 signing keys. Verification remains *possible* in software either way. +- [Observation volume grows the witness repo unboundedly] → Release-rate volume is low; MSTs handle millions of records; pointer retarget history (the chatty case) is deferred. +- [Measurement labels read as accusations] → Naming discipline (`release-modified`, never `compromise-suspected`) enforced at the spec level; severity is presented as consumer interpretation. + +## Migration Plan + +1. `service-account-migration` lands first (separate change): did:plc identity, single-account PDS, relay crawl, capabilities re-homed. +2. Witness writer ships observing from its start time; pre-existing releases receive `appeared` observations during a one-time backfill sweep with `observedAt` = sweep time (honest: "no later than" semantics hold; early releases simply have late first-seen times). +3. Label derivation and the resolve endpoint follow; the endpoint is additive (new query), nothing existing changes shape. +4. Rollback: the endpoint and labeler derivation can be disabled independently; observation records, once written, stay (append-only by design) — a halted witness is a stale witness, not a broken one. + +## Open Questions + +- Blob format versioning and layout details (header, offsets, multi-witness encoding for quorum) — settle during endpoint spec/implementation. +- Reference minimal verifier: does a C implementation live in this repo as a conformance artifact, or is the profile document alone the deliverable for this change? +- Backfill `observedAt` presentation: should backfilled observations carry a distinguishing marker (e.g. an event or field noting sweep origin) so consumers don't misread late first-seen times as late publication? +- Lexicon path convention: `town.dist.witness.observation` introduces a third path segment — confirm naming against existing flat `town.dist.*` convention before authoring the schema file. diff --git a/openspec/changes/release-witness/proposal.md b/openspec/changes/release-witness/proposal.md new file mode 100644 index 0000000..d26a12a --- /dev/null +++ b/openspec/changes/release-witness/proposal.md @@ -0,0 +1,44 @@ +# Release Witness + +## Why + +**dist.town's releases become evidence, not claims — verifiable by anyone, on anything, without trusting dist.town.** + +Today, a consumer who resolves a release trusts dist.town's read plane to have relayed the right record. The substrate never required that: atproto repos are signed Merkle trees, so every release record is already provable against nothing but the publisher's DID. This change closes the gap between what the substrate guarantees and what consumers can use, and adds the one guarantee the substrate cannot provide alone. + +1. **A witness that makes key compromise slow, public, and survivable — instead of instant and silent.** Self-certification proves *who signed*, never *whether the signer was still the publisher*. The witness adds what signatures can't: trusted time and observed history. Every release is countersigned with its first-seen moment; mutations of records that should never mutate become signed, timestamped, neutral measurements instead of prose norms. A consumer requiring "publisher-signed AND witnessed N hours ago AND not yanked" turns a stolen publisher key from a skeleton key into a race the attacker runs in public, against a yank button, on a visible firehose. No registry in the npm/PyPI/crates lineage offers this; it's certificate-transparency-shaped protection at package-registry cost. + +2. **Accountability that includes dist.town itself.** The witness's observations live as records in an ordinary atproto repo, under the same immutability norms it enforces on publishers — anyone mirroring the firehose can catch it equivocating. The witness role is defined by lexicon, not owned by dist.town: any operator can run one, and consumers can require quorum. dist.town's trust position is deliberately weak: it cannot forge a release, and it cannot suppress one silently. dist.town does not sell observability — it provides observation, and observation is checkable. + +3. **The proof point: the verification floor is low enough for a toy.** The full guarantee — authentic, witnessed, mature, current — compresses to a few KB of hash preimages and a handful of ECDSA verifies: no TLS stack, no CA store, no clock, no trusted channel. Firmware self-update on sub-ESP32-class hardware is the demonstration that the trust chain has no hidden dependency on powerful clients — if a Cortex-M0 can check dist.town's work, anything can. + +The one-line version: dist.town stops asking to be trusted, and starts being the reason nobody has to. + +## What Changes + +- New lexicon collection `town.dist.witness.observation`: signed, timestamped observations of publisher records, written to the witness's own repo. Events `appeared`/`mutated`/`deleted` (neutral measurement vocabulary), strongRef subject (`uri` + `cid`), `priorCid` on mutation/deletion, `repoRev`, `observedAt`. No prior-observation chain links — history derives from TID order; the repo's signed commit chain already provides ordering and tamper evidence. +- The witness observes **every** `town.dist.release` on the firehose (universal coverage, transparency-log posture), plus mutation/deletion of release and status records — the two record types whose existing norms declare them immutable/append-only. Pointer retarget history is deferred. +- The existing labeler derives measurement-named labels (e.g. `release-modified`) from observation records. Records are the verifiable source of truth; labels are the distribution rail. Naming stays neutral: modified is not compromised; severity interpretation belongs to consumers. +- New device-facing XRPC endpoint returning `application/octet-stream`: transcodes the publisher proof (record + MST path + signed commit, per `com.atproto.sync.getRecord`) and the witness proof into a flat, fixed-offset blob of hash preimages. The server does all parsing; the device does a SHA-256 chain plus ECDSA verifies. A malicious server can only DoS, never forge. A device-supplied nonce is countersigned with current lifecycle/pointer state for clock-free freshness. +- Documented device policy primitives: maturity window (`observedAt` + device-chosen N), `lastSeenTid` anti-rollback (string compare), pinned publisher DID + per-fleet pointer name (channels are pointers — no new publisher records), and the re-release norm for legitimate rollback past anti-rollback. +- A minimal-verifier profile documenting exactly what a constrained implementation must check, plus P-256 signing-key guidance for publishers targeting embedded consumers (cheap secure elements do P-256 natively; secp256k1 support in embedded silicon is rare). +- Witness role is defined, not owned: the lexicon is the interop contract; third-party witnesses write the same collection in their own repos; consumers may require quorum. + +## Capabilities + +### New Capabilities + +- `witness-observation`: The observation record — schema, event vocabulary, clock semantics (`observedAt` as the semantic field, TID as storage order, never backfilled from publisher-supplied data), universal release coverage, mutation/deletion detection for release and status records, and the defined-not-owned operator contract (third-party witnesses, quorum consumption). +- `device-update-resolution`: The device-facing resolve endpoint — transcoded dual-proof blob format, nonce-based clock-free freshness, and the device policy primitives (maturity window, TID anti-rollback, DID + fleet-pointer pinning, re-release norm) with the minimal-verifier profile and publisher key guidance. + +### Modified Capabilities + +- `release-moderation`: The labeler's vocabulary grows measurement-derived labels (e.g. `release-modified`) emitted from witness observation records — mechanical observations, named neutrally, distinct from judgment labels like `malware`. + +## Impact + +- **Lexicons**: new `lexicons/town/dist/witness/observation.json` (or equivalent path); new query lexicon for the device resolve endpoint. +- **AppView/indexer**: firehose consumer gains a witness writer (per-subject last-seen CID state it already needs for mutation detection doubles as `priorCid` source); label derivation from observation records; the new XRPC endpoint with proof transcoding. +- **Dependency**: requires the separate `service-account-migration` change — dist.town needs a did:plc account with its own repo (single-account PDS) so witness records have somewhere to live and devices can pin a rotation-capable identity (did:web has no self-certifying rotation story). This change is blocked on that one. +- **Docs**: minimal-verifier profile, device integration guide, publisher guidance (P-256, re-release norm). +- **No publisher-facing record changes**: releases, pointers, and status records are untouched; fleet channels reuse `town.dist.pointer` as-is. diff --git a/openspec/changes/release-witness/specs/device-update-resolution/spec.md b/openspec/changes/release-witness/specs/device-update-resolution/spec.md new file mode 100644 index 0000000..1b8bbcf --- /dev/null +++ b/openspec/changes/release-witness/specs/device-update-resolution/spec.md @@ -0,0 +1,67 @@ +# device-update-resolution Specification (delta) + +## ADDED Requirements + +### Requirement: Transcoded dual-proof resolve endpoint +The system SHALL provide an XRPC query that resolves a (publisher DID, project, pointer name) to the current target release and returns `application/octet-stream`: a flat, fixed-offset, versioned binary blob containing the publisher proof (release record bytes, MST path, signed commit — the content of `com.atproto.sync.getRecord`), the witness proof (observation record bytes, MST path, signed commit from the witness repo), and the PLC operation chains for both identities (deliverable incrementally from a client-supplied last-verified op index). The blob SHALL be structured as hash preimages with stated offsets such that a verifier performs only SHA-256 over supplied bytes, offset comparisons, and ECDSA verifies — no CAR, DAG-CBOR, or JSON parsing. + +#### Scenario: Device verifies with hashing and signature checks only +- **WHEN** a device receives a resolve blob for its pinned (DID, project, pointer) +- **THEN** it can establish "this release record is in the publisher's signed repo AND the witness observed these exact bytes" using only SHA-256, offset comparison, and ECDSA verification against keys derived from the pinned DIDs' PLC chains + +#### Scenario: Two proof chains meet at the record CID +- **WHEN** the device hashes the release record bytes from the publisher proof +- **THEN** the resulting CID both anchors the publisher's MST path and equals the witness observation's `subject.cid` + +### Requirement: The server is untrusted +The endpoint SHALL be constructed so that no response can cause a verifying client to accept tampered content: every trust-bearing byte in the blob is covered by the publisher's or witness's signature chain. A malicious or compromised endpoint can at most deny service (malformed or missing blobs). Clients MUST reject any blob whose hash chain or signatures fail, and MUST NOT fall back to unverified fields. + +#### Scenario: Forged blob fails closed +- **WHEN** an endpoint returns a blob whose release record bytes were altered +- **THEN** the record hash mismatches the MST path (or the witness `subject.cid`), verification fails, and the device keeps its current firmware + +### Requirement: Nonce-based clock-free freshness +The resolve request SHALL accept a client-supplied nonce. The response SHALL include a witness signature over (target release identity, current lifecycle state derived from status records, current pointer target, nonce), made with a dedicated verification method declared in the witness's DID document — not the repo signing key used raw. A valid nonce signature proves the answer is fresh without any client clock. Mutable facts ("not yanked now", "current target") SHALL be conveyed only via this channel, never as static records or labels. + +#### Scenario: Replayed answer is rejected +- **WHEN** an attacker replays a previously valid resolve response to a device that issued a new random nonce +- **THEN** the nonce signature does not cover the new nonce and the device rejects the response + +#### Scenario: Yanked release stops resolving fresh +- **WHEN** the current pointer target has a live `yanked` status record +- **THEN** the nonce-signed answer reports the target as not acceptable, and a conforming device does not install it + +### Requirement: Anti-rollback via TID comparison +Conforming device verifiers SHALL persist the TID rkey of the last accepted release and MUST reject any resolved release whose TID orders at or below it. TID rkeys are strings whose lexicographic order is creation order; the comparison is a string compare. + +#### Scenario: Stale-but-valid proof cannot downgrade +- **WHEN** a device holding lastSeenTid for v1.3 receives a fully valid blob for the older v1.2 release +- **THEN** the TID comparison fails and the device refuses the downgrade + +### Requirement: Maturity window as device policy +The blob SHALL carry the witness's `observedAt` for the target release so that verifiers can enforce a minimum age ("witnessed at least N ago") before accepting. The window length is device policy, chosen per fleet; the witness and endpoint MUST NOT impose one. The device-facing contract is: authentic (publisher proof) AND witnessed ≥ N ago (maturity) AND currently acceptable (nonce answer). + +#### Scenario: Freshly published release is deferred by policy +- **WHEN** a fleet configured with a 72-hour window resolves a release witnessed 2 hours ago +- **THEN** the device declines to install and retries later, without treating the response as an error + +### Requirement: Fleet channels are pointers +Device targeting SHALL reuse `town.dist.pointer` unchanged: a device pins (publisher DID, project, pointer name) at manufacture, and publishers steer fleets by retargeting per-fleet pointers (e.g. `fleet-mk3`). The system MUST NOT introduce a device-specific channel record type. + +#### Scenario: Fleet steered post-manufacture +- **WHEN** a publisher retargets the `fleet-mk3` pointer to a held-back release +- **THEN** devices pinning that pointer resolve the new target on next check, with no device-side configuration change + +### Requirement: Re-release norm for legitimate rollback +Publisher documentation SHALL establish the re-release norm: to roll a fleet back past anti-rollback, publish a new release record (new TID) referencing the previous artifacts' digests. Retargeting a pointer at an older release MUST be documented as ineffective for devices that already accepted a newer TID. + +#### Scenario: Bad release recovered without bricking anti-rollback +- **WHEN** a publisher yanks a bad release and publishes a re-release of the prior artifacts under a new TID +- **THEN** devices that accepted the bad release's TID accept the re-release, and the yanked release stops resolving + +### Requirement: Minimal-verifier profile +The system SHALL publish a normative verifier profile documenting exactly what a constrained implementation must check (hash chain, both signature chains, PLC op verification, CID meeting point, TID anti-rollback, maturity window, nonce freshness), the blob format version contract, and publisher guidance: publishers targeting embedded consumers SHOULD use P-256 (not secp256k1) atproto signing keys, because commodity secure elements and MCU crypto accelerators support P-256 natively. + +#### Scenario: Independent implementation from the profile alone +- **WHEN** a device vendor implements a verifier from the profile document without reading dist.town source code +- **THEN** it accepts exactly the blobs a conforming verifier accepts and rejects tampered, stale, immature, and downgraded responses diff --git a/openspec/changes/release-witness/specs/release-moderation/spec.md b/openspec/changes/release-witness/specs/release-moderation/spec.md new file mode 100644 index 0000000..3f78478 --- /dev/null +++ b/openspec/changes/release-witness/specs/release-moderation/spec.md @@ -0,0 +1,14 @@ +# release-moderation Specification (delta) + +## ADDED Requirements + +### Requirement: Measurement labels derived from witness observations +The labeler SHALL emit measurement labels (vocabulary including `release-modified`, `release-deleted`, `status-modified`) derived from witness observation records. Each measurement label corresponds to an observation record, which remains the verifiable source of truth; labels are the distribution rail reaching subscribed surfaces. Measurement label names MUST use neutral, mechanical language describing what was observed — never judgment or severity terms (no `compromise-suspected` or similar): a modified release is not necessarily a compromised one, and severity interpretation belongs to consumers. Measurement labels SHALL be documented as distinct from judgment labels (`malware`, `takedown`, `spam`): consumers MAY auto-act on measurements while treating judgments as advisory. + +#### Scenario: Mutation observation surfaces through label rails +- **WHEN** the witness writes a `mutated` observation for a release record +- **THEN** the labeler emits `release-modified` on that release, and subscribed consumer surfaces show the warning with the observation record available as the verifiable evidence behind it + +#### Scenario: Measurement naming stays neutral +- **WHEN** a new measurement-derived label value is added to the vocabulary +- **THEN** its name describes the observed event mechanically and contains no judgment of intent or compromise diff --git a/openspec/changes/release-witness/specs/witness-observation/spec.md b/openspec/changes/release-witness/specs/witness-observation/spec.md new file mode 100644 index 0000000..ea248e3 --- /dev/null +++ b/openspec/changes/release-witness/specs/witness-observation/spec.md @@ -0,0 +1,67 @@ +# witness-observation Specification (delta) + +## ADDED Requirements + +### Requirement: Observation record schema +The system SHALL define a `town.dist.witness.observation` record (TID key) written to the witness's own repo, containing: `event` (string, known values `appeared`, `mutated`, `deleted`; open vocabulary — records with unrecognized values are ignored for state derivation, not rejected), `subject` (strongRef: `uri` + `cid` of the observed record as observed), `observedAt` (datetime, required), `repoRev` (string, the subject repo's commit rev at observation), and `priorCid` (cid, present on `mutated` and `deleted` events: the subject CID previously observed). Event names MUST be neutral measurement vocabulary; the schema MUST NOT encode judgment terms (e.g. no `compromised`). + +#### Scenario: Appeared observation pins exact bytes +- **WHEN** a new `town.dist.release` record appears on the firehose +- **THEN** the witness writes an observation with `event: appeared` and `subject.cid` equal to the CID of the record bytes as observed, so any holder of the release bytes can hash them and compare + +#### Scenario: Mutation observation carries both CIDs +- **WHEN** a record at an already-observed release at-uri appears with a different CID +- **THEN** the witness writes an observation with `event: mutated`, `subject.cid` set to the new CID, and `priorCid` set to the previously observed CID + +### Requirement: Observations are independent records with no chain links +Observation records MUST NOT reference other observation records (no prior-observation links). Per-subject history SHALL be derived by indexers scanning observations by `subject.uri` in TID order. Duplicate observations of the same event (e.g. from at-least-once firehose processing) are valid and MUST be tolerated by consumers as identical measurements. + +#### Scenario: Replayed event produces a harmless duplicate +- **WHEN** the witness reprocesses a firehose event it already observed and writes a second identical observation +- **THEN** both records are valid, and indexers deduplicate by (subject, event, cid) without any chain repair + +### Requirement: Universal release coverage +The witness SHALL observe every `town.dist.release` record on the firehose with an `appeared` observation, with no opt-in, registration, or project filtering. Coverage gaps are themselves observable defects. + +#### Scenario: Release observed without any publisher action +- **WHEN** a publisher who has never interacted with dist.town creates a release record +- **THEN** an `appeared` observation for it is written to the witness repo + +### Requirement: Mutation and deletion detection for immutable-by-norm records +The witness SHALL emit `mutated` and `deleted` observations for `town.dist.release` and `town.dist.status` records — the record types whose existing norms declare them immutable and append-only respectively. Pointer retargeting MUST NOT produce observations (it is expected behavior per the pointer lexicon); pointer history is out of scope. + +#### Scenario: Status record deletion is observed +- **WHEN** a `town.dist.status` record is deleted from a publisher repo +- **THEN** the witness writes an observation with `event: deleted` and `priorCid` set to the last observed CID of that status record + +#### Scenario: Pointer move produces no observation +- **WHEN** a publisher retargets a `town.dist.pointer` record +- **THEN** no observation record is written + +### Requirement: observedAt clock semantics +`observedAt` SHALL be the witness's own firehose receipt time, meaning the subject existed no later than that moment. It MUST NOT be derived from or backfilled with publisher-supplied data (record contents, commit metadata, or claimed timestamps). The observation's TID rkey is storage order only and MAY skew from `observedAt` under batched writes; consumers MUST treat `observedAt` as the semantic time. + +#### Scenario: Backdated publisher claim does not move observedAt +- **WHEN** a release record containing an old-looking timestamp in its content appears on the firehose today +- **THEN** its `appeared` observation carries today's receipt time as `observedAt` + +### Requirement: Witness repo is subject to witness norms +Observation records SHALL be append-only: never edited, never deleted. The witness's repo flows through the relay/firehose like any repo, and mutation or deletion of an observation record is itself an anomaly detectable by any mirror — the witness is accountable under the same rules it applies to publishers. + +#### Scenario: Mirrors can detect witness misbehavior +- **WHEN** a third party mirroring the witness repo sees an observation record change CID between commits +- **THEN** the mirror has cryptographic evidence of witness misbehavior, signed by the witness's own key + +### Requirement: The witness role is defined, not owned +The observation lexicon SHALL be the complete operator contract: any party MAY run a witness by writing `town.dist.witness.observation` records to its own repo under its own DID, and nothing in the protocol SHALL privilege dist.town's witness. Consumers MAY pin multiple witness DIDs and require agreement (quorum) on `subject.cid` before accepting a release. + +#### Scenario: Third-party witness interoperates without coordination +- **WHEN** an independent operator writes conforming observation records in its own repo +- **THEN** a consumer pinning that operator's DID verifies its observations with the same machinery used for dist.town's, with no dist.town involvement + +### Requirement: Backfill on witness start +When the witness begins operating (or recovers from an outage), it SHALL sweep existing `town.dist.release` records and write `appeared` observations with `observedAt` set to the sweep time. Backfilled observations are honest under "no later than" semantics; the witness MUST NOT fabricate earlier observation times. + +#### Scenario: Pre-existing release gets a late first-seen time +- **WHEN** the witness starts and sweeps a release published a year earlier +- **THEN** the `appeared` observation carries the sweep time, not the release's claimed age diff --git a/openspec/changes/release-witness/tasks.md b/openspec/changes/release-witness/tasks.md new file mode 100644 index 0000000..9c545f6 --- /dev/null +++ b/openspec/changes/release-witness/tasks.md @@ -0,0 +1,40 @@ +# Tasks: Release Witness + +## 1. Prerequisites and lexicons + +- [ ] 1.1 Confirm `service-account-migration` change has landed (did:plc identity, single-account PDS, relay crawl) — this change is blocked without it +- [ ] 1.2 Resolve the lexicon path convention question (nested `town.dist.witness.observation` vs. flat naming) and author the observation record lexicon per the witness-observation spec +- [ ] 1.3 Author the device resolve query lexicon (params including nonce and last-verified PLC op indexes; `application/octet-stream` output) and validate both lexicons +- [ ] 1.4 Specify the blob format: version header, fixed-offset preimage layout, PLC op chain framing, witness `observedAt` field, nonce signature framing — written up as the normative section of the verifier profile + +## 2. Witness writer + +- [ ] 2.1 Add per-subject last-seen CID state to the firehose indexer (keyed by subject at-uri) for release and status records +- [ ] 2.2 Write `appeared` observations for every `town.dist.release` on the firehose, with `observedAt` = receipt time, `repoRev`, and strongRef subject +- [ ] 2.3 Write `mutated`/`deleted` observations (with `priorCid`) for release and status records; verify pointer events produce no observations +- [ ] 2.4 Implement the backfill sweep on witness start/recovery (`observedAt` = sweep time), idempotent under re-runs +- [ ] 2.5 Verify duplicate-tolerance end to end: replaying firehose events yields harmless duplicate observations and stable derived state + +## 3. Label derivation + +- [ ] 3.1 Emit `release-modified`, `release-deleted`, `status-modified` labels from observation records through the existing labeler +- [ ] 3.2 Surface measurement labels in consumer UI with the observation record linked as evidence, visually distinct from judgment labels + +## 4. Resolve endpoint + +- [ ] 4.1 Implement proof assembly: fetch publisher `sync.getRecord` proof, witness observation proof, and both PLC op chains; transcode to the blob format +- [ ] 4.2 Implement nonce countersigning with a dedicated verification method key (declared in the witness DID document; domain-separated from repo signing) over (release identity, lifecycle state, pointer target, nonce) +- [ ] 4.3 Wire pointer resolution + status derivation into the fresh answer; verify yanked targets report not-acceptable +- [ ] 4.4 Fail-closed behavior: malformed upstream data produces an error response, never a partial blob + +## 5. Verifier profile and validation + +- [ ] 5.1 Write the minimal-verifier profile doc: required checks (hash chain, signatures, PLC ops, CID meeting point, TID anti-rollback, maturity window, nonce freshness), blob version contract, P-256 publisher guidance, re-release norm +- [ ] 5.2 Build a reference verifier (host-side test harness is sufficient for this change) that implements the profile with only SHA-256 + ECDSA primitives and no CAR/CBOR parsing +- [ ] 5.3 Adversarial tests against the reference verifier: tampered record bytes, wrong witness subject CID, replayed nonce answer, downgraded TID, immature release, forged blob from a hostile server — all rejected +- [ ] 5.4 End-to-end test: publish → witness observes → resolve → verify → yank → fresh answer flips → re-release → anti-rollback accepts new TID + +## 6. Documentation + +- [ ] 6.1 Publisher docs: P-256 signing keys for embedded consumers, per-fleet pointers, re-release norm +- [ ] 6.2 Operator docs: running a third-party witness from the lexicon contract alone; quorum consumption guidance diff --git a/openspec/changes/service-account-migration/.openspec.yaml b/openspec/changes/service-account-migration/.openspec.yaml new file mode 100644 index 0000000..9567240 --- /dev/null +++ b/openspec/changes/service-account-migration/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-08-18 diff --git a/openspec/changes/service-account-migration/design.md b/openspec/changes/service-account-migration/design.md new file mode 100644 index 0000000..0fa148a --- /dev/null +++ b/openspec/changes/service-account-migration/design.md @@ -0,0 +1,63 @@ +# Design: Service Account Migration + +## Context + +The service-identity spec establishes `did:web:dist.town` as the single identity for admin auth, moderation reports, and label signing, with its DID document edge-served by the AppView. This worked because none of those capabilities needed a repo or long-horizon key pinning. The `release-witness` change introduces both needs, and `public-deployment` (14/15 tasks) creates a closing window: identities referenced by emitted labels are permanent. + +## Goals / Non-Goals + +**Goals:** +- A rotation-capable, repo-bearing service identity all capabilities share. +- Preserve the one-identity principle and the human-readable `dist.town` name. +- Land before public references to did:web harden. + +**Non-Goals:** +- Witness functionality itself (separate change, blocked on this one). +- General-purpose PDS hosting for anyone but the service account. +- Publishing dist.town's own projects from the account (possible later; nothing here precludes it). +- The web console with atproto OAuth login (deferred; see D5 — it is an OAuth client at the AppView layer and needs nothing from the PDS). + +## Decisions + +### D1: did:plc, not did:web +did:web has no self-certifying rotation: verifying its current keys means trusting a TLS fetch of `did.json`, which is exactly the trust the witness/device story eliminates. did:plc's operation chain is a signature chain verifiable offline from bytes alone, so devices can pin the DID for a product lifetime and verify rotations incrementally. The PLC directory dependency is accepted: the chain is self-certifying; the directory is discovery, not a trust root. + +### D2: The identity is an account with handle `dist.town` +Rather than a bare DID, the service becomes an ordinary atproto account: did:plc with the handle `dist.town` (verified via the domain the service already controls). Human-readable naming is preserved through the standard handle mechanism instead of the DID method, and the account model is what gives the identity a repo. + +### D3: Minimal custom PDS implementation, in Go, in this repo +Self-hosted PDS running exactly one account: repo storage, `com.atproto.sync.*`, firehose emission, and a relay crawl request so the ecosystem indexes it. No user OAuth, no app passwords, no multi-tenancy. Hosting the account on a third-party PDS was rejected (repo custody should not depend on an external operator); the reference PDS was rejected in favor of a minimal custom implementation — the dependency inventory makes assembly cheaper than operating a full TypeScript PDS for one account. + +The substrate is largely already in `go.mod`: indigo (July 2026 snapshot), `whyrusleeping/cbor-gen`, `go-cid`, the ipfs blockstore packages, `gorilla/websocket`, and `modernc.org/sqlite` (the block store). indigo's `atproto/repo` + `atproto/repo/mst` provide the core write primitives — `mst.Tree` with `ApplyOp`, `Repo.Commit()` producing an unsigned `Commit`, `Commit.Sign`/`VerifySignature`, `NormalizeOps`/`CheckOp`/`InvertOp`, `LoadRepoFromCAR` — with `atproto/crypto` for k256/p256 keys, the indigo `events` package for subscribeRepos machinery, and PLC operation authoring patterns in `cmd/goat` (ops are DAG-CBOR + low-S ECDSA; every primitive is present). + +Known caveat to de-risk first: the `atproto/repo` docs state the package "does not yet work for implementing a repository host (PDS)." The gap is host-side machinery — block storage management, CAR export and `getRecord` proof slices, sync endpoint serving, firehose emission — not the MST/commit core. An early spike validates the write path, CAR export, and proof-slice generation against the pinned indigo version and identifies what gets written in-repo vs. upstreamed. Architecture references: haileyok/cocoon (Go PDS, SQLite block+blob store, experimental) and millipds (Python). + +### D4: One identity survives; did:web becomes a deprecated alias +The service-identity principle ("no capability mints a separate identity") is kept — every capability re-homes to the did:plc account. `/.well-known/did.json` continues to be served, now as a deprecated pointer document (`alsoKnownAs` linking the did:plc identity in both directions) so existing di:web references resolve to the successor rather than dangling. No new did:web references are minted after cutover. + +### D5: The PDS stays OAuth-free; the future web console does not touch it +A dist.town web console with atproto OAuth delegated login (e.g. admins reviewing moderation reports) was considered for this change and explicitly does not alter PDS scope: such a console is an OAuth **client**, authenticating admins against *their own* PDSes — indigo ships `atproto/auth/oauth` with a client-focused `ClientApp`, so the path is well supported in Go when wanted. The single-account PDS therefore needs no OAuth authorization server, no user accounts, nothing beyond D3's surface. The only coupling to this migration is the admin-auth audience DID switch already planned; mapping logged-in DIDs to admin roles is AppView-layer future work, deferred and unblocked. + +### D6: Rotation key custody is part of the change, not an afterthought +The PLC rotation keys become the decade-scale root of trust. Custody (offline storage, who holds them, recovery procedure) is documented as a deliverable of this change. The repo signing key (online, in the PDS) and rotation keys (offline) are distinct by PLC design; the witness's later nonce-signing key will be a third, declared verification method — all delivered to verifiers through the same op chain. + +## Risks / Trade-offs + +- [PLC directory unavailability or misbehavior] → Op chain is self-certifying; consumers verify from bytes; directory outage degrades discovery, not trust. +- [Rotation key loss = permanent identity loss] → Documented custody with offline backup and a recovery drill before any device-pinning consumers exist. +- [Labels already emitted under did:web] → Pre-launch volume is near zero; cutover re-signs nothing — old labels remain historically valid under the deprecated alias, and the transition doc states the succession. +- [Running a PDS adds an always-on service] → Minimal single-account surface; a down PDS stalls witness writes (future) but affects no current capability except label signing key resolution, which caches. +- [indigo's repo package is documented as not yet host-ready] → The gap is host machinery, not the MST/commit core; the spike task validates the write path first, and cocoon/millipds provide working architecture references if pieces must be written from scratch. + +## Migration Plan + +1. Create the PLC account; verify handle `dist.town`; establish rotation key custody. +2. Stand up the single-account PDS; request relay crawl; confirm the repo appears on the firehose. +3. Re-home capabilities in one deploy: labeler signing key/endpoint declared in the did:plc document, `createReport` subject and admin JWT audience switched. +4. Swap `/.well-known/did.json` to the deprecated-alias document. +5. Rollback (before step 3 completes): capabilities still function under did:web; the plc account existing early is harmless. + +## Open Questions + +- Whether the alias `did:web` document ever gets retired outright, or persists indefinitely as succession evidence. +- Spike outcome: which host-side pieces (CAR export, proof slices, firehose emission) land in-repo vs. as indigo upstream contributions. diff --git a/openspec/changes/service-account-migration/proposal.md b/openspec/changes/service-account-migration/proposal.md new file mode 100644 index 0000000..6a0278e --- /dev/null +++ b/openspec/changes/service-account-migration/proposal.md @@ -0,0 +1,30 @@ +# Service Account Migration + +## Why + +dist.town's service identity is `did:web:dist.town` — an identity with no repo and no self-certifying key rotation: its keys are whatever `did.json` says today, verifiable only by trusting a TLS fetch. The witness role planned in `release-witness` needs both things did:web cannot provide — a repo for observation records, and an identity that constrained devices can pin for a decade and verify rotations of offline (did:plc's operation chain is a signature chain; did:web's rotations are unverifiable). Timing makes this urgent independent of the witness work: `public-deployment` is at 14/15 tasks, and every day after launch `did:web:dist.town` accretes permanent references (labels carry their emitter forever, consumers pin identities, docs cite them). The migration is nearly free this week and genuinely painful in six months. + +## What Changes + +- dist.town becomes a first-class atproto account: a did:plc identity with the handle `dist.town`, hosted on its own single-account PDS — a minimal custom Go implementation (repo storage, `com.atproto.sync.*`, firehose emission, relay crawl — no user OAuth, no multi-tenancy, no blob serving). A future admin web console with atproto OAuth login was considered and confirmed out of PDS scope: it would be an OAuth client at the AppView layer. +- **BREAKING** All service capabilities re-home to the did:plc identity: the labeler signs as it, `createReport` addresses it, admin service auth audiences it. The service-identity principle — one identity, no capability mints its own — is preserved; only the DID method changes. +- `did:web:dist.town` enters a documented transition: its DID document remains served and points to the did:plc account (`alsoKnownAs`), marked deprecated; no new references are minted. +- PLC rotation key custody is established and documented — these keys become the long-term root of trust that devices and consumers pin. + +## Capabilities + +### New Capabilities + +_None — this reshapes the existing service identity._ + +### Modified Capabilities + +- `service-identity`: the service DID requirement changes from `did:web:dist.town` to a did:plc account with handle `dist.town`; new requirements for the service repo/PDS, rotation key custody, and the did:web transition. + +## Impact + +- **Infrastructure**: a minimal custom single-account PDS implemented in Go in this repo (the substrate — indigo's `atproto/repo`/`mst`/`crypto`/`events` packages, cbor-gen, SQLite — is already in `go.mod`; an early spike validates indigo's host-side gaps), relay crawl request, PLC account creation, handle verification for `dist.town`. +- **Moderation pipeline**: labeler signing identity and declared service endpoint move to the did:plc document; `createReport` subject and admin JWT audience configuration change. +- **AppView**: `/.well-known/did.json` keeps serving, now as the deprecated did:web alias document. +- **Downstream**: `release-witness` is blocked on this change; no publisher-facing records or lexicons are affected. +- **Timing**: should land before (or with) `public-deployment` completing, while `did:web:dist.town` references are still cheap to abandon. diff --git a/openspec/changes/service-account-migration/specs/service-identity/spec.md b/openspec/changes/service-account-migration/specs/service-identity/spec.md new file mode 100644 index 0000000..2ba2633 --- /dev/null +++ b/openspec/changes/service-account-migration/specs/service-identity/spec.md @@ -0,0 +1,42 @@ +# service-identity Specification (delta) + +## MODIFIED Requirements + +### Requirement: dist.town has a service DID + +The service SHALL identify itself as a did:plc account bearing the handle `dist.town` (verified via the domain the service controls). This identity is the audience for admin service auth, the subject services report to (`createReport`), and the emitter the labeler signs as — one identity, stated once; no capability mints a separate identity. The former `did:web:dist.town` document remains served at `/.well-known/did.json` as a deprecated alias linking the did:plc identity via `alsoKnownAs`, and no new `did:web` references are minted after cutover. + +#### Scenario: The identity resolves as an ordinary account + +- **WHEN** a client resolves the handle `dist.town` +- **THEN** it obtains the service's did:plc identity and DID document through standard atproto identity resolution + +#### Scenario: One audience across capabilities + +- **WHEN** an admin JWT, a moderation report, or a label names the dist.town service +- **THEN** each uses the service's did:plc DID — no capability mints a separate identity + +#### Scenario: Legacy did:web references resolve to the successor + +- **WHEN** a client fetches `https://dist.town/.well-known/did.json` +- **THEN** a valid DID document is returned that identifies the did:plc account as the successor identity via `alsoKnownAs` + +## ADDED Requirements + +### Requirement: The service identity has a repo + +The service account SHALL be hosted on a dist.town-operated single-account PDS providing repo storage, the `com.atproto.sync.*` endpoints, and firehose emission, and SHALL be crawled by a relay so its repo propagates through the ecosystem. The PDS MUST NOT offer user account creation, user OAuth, or multi-tenant hosting. + +#### Scenario: Service records reach the firehose + +- **WHEN** a record is written to the service account's repo +- **THEN** it appears in the relay firehose and is verifiable against the service's signed commit chain like any publisher's record + +### Requirement: Rotation key custody + +The PLC rotation keys for the service identity SHALL be held offline, separate from the PDS's online repo signing key, with documented custody and a tested recovery procedure. Key rotation history remains verifiable by third parties through the self-certifying PLC operation chain. + +#### Scenario: Signing key rotation is externally verifiable + +- **WHEN** the service rotates its repo signing key via a PLC operation +- **THEN** any consumer holding the operation chain can verify the rotation offline, from the chain's signatures alone, without trusting the PLC directory or a TLS fetch diff --git a/openspec/changes/service-account-migration/tasks.md b/openspec/changes/service-account-migration/tasks.md new file mode 100644 index 0000000..5e5cc0a --- /dev/null +++ b/openspec/changes/service-account-migration/tasks.md @@ -0,0 +1,23 @@ +# Tasks: Service Account Migration + +## 1. Identity + +- [ ] 1.1 Create the did:plc account and verify the handle `dist.town` +- [ ] 1.2 Establish rotation key custody: offline storage, documented holders and recovery procedure, recovery drill performed + +## 2. PDS + +- [ ] 2.1 Spike: validate the repo write path against the pinned indigo version — MST mutation via `ApplyOp`, `Commit()` + `Sign`, CAR export, and `getRecord` proof-slice generation; record which host-side pieces must be written in-repo vs. upstreamed +- [ ] 2.2 Implement the minimal single-account PDS in Go: SQLite block store, repo write path, `com.atproto.sync.*` endpoints, subscribeRepos firehose emission (no user account creation, no OAuth) +- [ ] 2.3 Request relay crawl and confirm the service repo propagates through the firehose + +## 3. Capability re-homing + +- [ ] 3.1 Declare the labeler service endpoint and label signing key in the did:plc document; switch the labeler to sign as the did:plc identity +- [ ] 3.2 Switch `createReport` subject and admin service-auth audience to the did:plc DID +- [ ] 3.3 Replace `/.well-known/did.json` with the deprecated-alias document (`alsoKnownAs` succession in both directions) + +## 4. Verification and docs + +- [ ] 4.1 End-to-end check: handle resolution, label signature verification against the did:plc document, report submission, admin auth — all under the new identity +- [ ] 4.2 Document the identity succession (old did:web → did:plc) and update any references in docs and deployment config