diff --git a/LOOP_STATE.md b/LOOP_STATE.md index f57bdd9..fb09b90 100644 --- a/LOOP_STATE.md +++ b/LOOP_STATE.md @@ -8,7 +8,7 @@ update this file → schedule next. Stop the loop when every task is `done`. |---|------|--------|--------|-------| | 1 | 01-scaffold-storage | done | (see git log) | reviewed by 7 reviewers, 18 fixes applied | | 2 | 02-ap-protocol | done | (see git log) | 5 reviewers incl. security; 14 fixes (critical: actor-id binding; high: SSRF, webfinger host confusion) | -| 3 | 03-identity-repos | pending | | | +| 3 | 03-identity-repos | done | (see git log) | 8 reviewers (5 Claude + codex/gemini/glm); 16 fixes (genesis race, seq ordering, MST-corruption-as-NotFound, KeyUse deletes, TID micro-fill) + 7 new tests | | 4 | 04-sync-firehose | pending | | | | 5 | 05-materializer | pending | | | | 6 | 06-ingestion | pending | | | @@ -83,3 +83,94 @@ and deferred TODOs here) EVERY request incl. GET; keyId is {actorID}#main-key; hs2019 treated as rsa-sha256; 1h date-skew window. - .claude/ is gitignored (session/tooling state, incl. scheduled_tasks.lock). + +### From task 03 (identity + virtual repo layer — tasks 04/05/06 consume this) +- internal/repo.Manager is the ONLY write path into repos. PutRecord/ + DeleteRecord return (*repo.CommitResult, error) — {RecordCID (empty for + deletes), CommitCID, Rev, Seq, NoOp}. NewManager returns (*Manager, + error) (nil db/keys rejected). Records must carry non-empty `$type`. + Identical re-put = idempotent NO-OP: NoOp=true, Seq=0, same cid+rev, + NO new commit/firehose event (deterministic rkeys rely on this). +- repo.DeterministicTID(published time.Time, canonicalAPID string) + (syntax.TID, error) is task 05's rkey function. FAILS CLOSED on zero/ + pre-epoch published (callers still gate on ap.Time.OK()). For second- + precision inputs the microsecond field is filled from sha256(ap_id) — + same-second bulk imports don't birthday-collide the 10 clock-ID bits; + within-second sort order is hash order. GOLDEN-VALUE TESTS pin the + algorithm (tid_test.go) — changing it breaks every persisted at-uri. + Commit revs come from repo_state via NextRev — monotonic per repo + across restarts. ops use typed repo.OpAction consts. +- firehose_events schema for task 04: seq bigserial, did, commit_cid, + prev_data_cid (MST root before commit, NULL on genesis — the sync v1.1 + prevData), since_rev (previous commit's rev, NULL on genesis — the + #commit `since` field), rev, ops jsonb ([{action,path,cid,prev}]), + car bytea (CARv1, ROOT/COMMIT BLOCK FIRST, contains commit + MST-diff + + record blocks), created_at. Appended in the SAME tx as the commit. + Commits are v3, Prev always null. +- Commit serialization: every commitWrite tx takes GLOBAL + pg_advisory_xact_lock(0x7469646570636d) (distinct from testutil's + session lock 0x7469646570 — keep them distinct). This guarantees + seq order == commit-visibility order, so task 04 may tail with naive + `WHERE seq > cursor` — any future writer bypassing repo.Manager breaks + that. Per-DID mutex + repo_state row lock remain as backstops. blocks + keeps superseded blocks (no GC; append-only is load-bearing for + GetRecord's read consistency); ExportCAR includes unreachable + historical blocks — task 04's getRepo may want reachable-set-only. +- identity.Minter.MintActor mints did:plc via MODERN plc_operation genesis + ops (indigo's plc package only has the deprecated legacy `create` op — + don't use it): rotationKeys=[bridge escrow key], verificationMethods. + atproto=per-actor key, signed by the escrow rotation key (enables later + claiming). Minter does NOT write bridged_actors — callers (05/06) upsert + the returned Identity{DID, Handle, DIDKey, SigningKeyEncrypted}; handle + uniqueness race is caught by the bridged_actors_handle_key index. +- Minting failure semantics (tasks 05/06): a failed mint can leave a + registered DID on the directory (PLC ops are forever); orphans are + slog.Error'd with did+handle. On handle-collision retry, callers should + eventually REUSE the minted DID via a PLC updateHandle op, not re-mint + (deferred). Unrepresentable usernames (all-CJK/emoji) get deterministic + u<10-hex-of-sha256> labels; collision suffixes shorten the base so the + 63-char DNS label limit holds. No mint rate limiting yet — REVISIT in + task 06 when inbound AP activity can trigger minting (abuse vector). +- Key custody: identity.Custodian (AES-256-GCM under 32-byte BRIDGE_KEK, + new env var, dev default is a fixed public key, required in prod). + Ciphertexts are AAD-bound to the DID — copying signing_key between rows + breaks decryption. Escrow rotation key lives ENCRYPTED in service_keys + row "plc-rotation" (unlike the plaintext RSA service key; NOTE the + column is named private_key_pem but holds sealed ciphertext — rename + candidate). identity.ActorKeys implements repo.SigningKeys — + SigningKey(ctx, did, use repo.KeyUse): tombstoned actor + KeyUseWrite → + IsTombstoned (frozen); tombstoned + KeyUseDelete → key RELEASED, so + task 05's Delete(Actor) → scrub-records flow works regardless of + consent-flip ordering. Residual TOCTOU: a consent flip racing an + in-flight commit can let that ONE commit land (consent read is outside + the commit tx — full fix deferred; fine for single-writer v1). +- store.BridgedActors grew GetByHandle (minting collision-suffix + + resolveHandle). Handle scheme: name.instance-with-dashes.BRIDGE_HOSTNAME, + lowercased, non-[a-z0-9-] runs → single dash, collisions get -2/-3/…. + Tombstoned actors' handles do NOT resolve. +- Wired in main.go: GET /xrpc/com.atproto.identity.resolveHandle and + GET /.well-known/atproto-did (resolves from Host header; wildcard DNS + requirement documented in README). Task 04 mounts sync endpoints next to + them. +- PLC egress uses ap.NewGuardedHTTPClient (same SSRF guard as the AP + client; ALLOW_PRIVATE_FETCH relaxes it in dev/tests only). +- Testing: internal/testutil.DB(t) is the shared pg harness — it holds a + postgres advisory lock per test process because store/repo/identity + packages share the test DB and `go test ./...` runs packages in parallel. + New pg-using packages MUST use it. PLC tests hit a LOCAL directory only + (default http://localhost:3002, env TIDEPOOL_TEST_PLC_URL, `make plc-up` + or the running Coves dev PLC); they hard-fail on non-loopback URLs and + skip under -short/unreachable. NEVER point tests at https://plc.directory. +- go.mod grew direct deps: go-cid, go-block-format, go-ipld-format, + go-car (v0, indigo's pinned pseudo-version), go-multihash. +- Test DB URL needs ?sslmode=disable (the bare URL earlier in this file + fails with "SSL is not enabled"); Makefile's TEST_DATABASE_URL has it. +- Deferred design notes for task 04: repo package should own the sync + read API (GetRecordProof for sync.getRecord, ListEvents(sinceSeq, + limit)) rather than task 04 issuing raw SQL against repo tables — + decide there. MST loads are full-tree, one SELECT per node → PutRecord + is O(repo size); fine now, needs a per-DID tree cache before big + community backfills (task 05). SigningKeys could become a SignCommit + capability (keeps key plaintext inside identity; enables KMS later) — + revisit before the interface calcifies. No OnCommit hook yet: task 04's + broadcaster should LISTEN/NOTIFY or poll seq (CommitResult.Seq exists). diff --git a/Makefile b/Makefile index 7207a81..fc9856c 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,4 @@ -.PHONY: help build run test test-db-up test-db-down db-migrate db-migrate-down dev-up dev-down lint fmt fmt-check clean +.PHONY: help build run test test-db-up test-db-down db-migrate db-migrate-down dev-up dev-down plc-up plc-down lint fmt fmt-check clean .DEFAULT_GOAL := help @@ -41,11 +41,20 @@ dev-up: ## Start the dev database (port 5442) @$(COMPOSE) up -d --wait postgres @echo "$(GREEN)✓ PostgreSQL (dev) on localhost:5442$(RESET)" -dev-down: ## Stop all dev services (including the test database) +dev-down: ## Stop all dev services (including the test database and PLC) @echo "$(YELLOW)Stopping Tidepool dev stack...$(RESET)" - @$(COMPOSE) --profile test down --remove-orphans + @$(COMPOSE) --profile test --profile plc down --remove-orphans @echo "$(GREEN)✓ Stopped$(RESET)" +plc-up: ## Start the local PLC directory (port 3002; first run builds did-method-plc) + @echo "$(GREEN)Starting local PLC directory (first run takes several minutes)...$(RESET)" + @$(COMPOSE) --profile plc up -d --wait postgres-plc plc-directory + @echo "$(GREEN)✓ PLC directory on http://localhost:3002$(RESET)" + +plc-down: ## Stop the local PLC directory + @$(COMPOSE) --profile plc stop plc-directory postgres-plc + @echo "$(GREEN)✓ PLC directory stopped$(RESET)" + ##@ Database Management db-migrate: ## Apply migrations to the dev database (goose CLI) diff --git a/README.md b/README.md index 146f80f..154a82e 100644 --- a/README.md +++ b/README.md @@ -23,5 +23,50 @@ Requires Go 1.25+, Docker, and (for `make db-migrate` / `make lint`) the `goose` and `golangci-lint` CLIs. Store tests need a real postgres: they skip with a clear message when `TIDEPOOL_TEST_DATABASE_URL` is unset. -Configuration is environment variables with logged dev defaults — see -`internal/config/config.go`. +The identity-minting tests additionally need a **local** PLC directory +(`make plc-up`, port 3002; the first start clones and builds +[did-method-plc](https://github.com/did-method-plc/did-method-plc), which +takes several minutes). They skip when it is unreachable and under +`go test -short`, and hard-refuse to run against any non-loopback +directory — the test suite can never create DIDs on the public +`plc.directory`. Point `TIDEPOOL_TEST_PLC_URL` elsewhere-on-localhost if +your directory is on a different port. + +## Configuration + +Environment variables with logged dev defaults (see +`internal/config/config.go`); everything below is **required in +production**: + +| Variable | Dev default | Meaning | +|---|---|---| +| `DATABASE_URL` | local dev postgres | bridge state | +| `LISTEN_ADDR` | `:8091` | HTTP bind address | +| `BRIDGE_HOSTNAME` | `localhost` | public domain of the bridge; anchors handles and the PDS endpoint in minted DID docs | +| `PLC_DIRECTORY_URL` | `http://localhost:3002` (local, `make plc-up`) | did:plc directory; production uses `https://plc.directory` | +| `BRIDGE_KEK` | fixed public dev key | 32-byte key-encryption key (64 hex chars or base64) sealing per-actor signing keys and the escrow rotation key at rest (AES-256-GCM) | +| `BRIDGE_SERVICE_DID` | *(optional)* | pre-provisioned service DID for the bridge's own actor | +| `USER_AGENT` | derived | outbound HTTP user agent | +| `ALLOW_PRIVATE_FETCH` | off | dev-only: disables the SSRF egress guard (AP fetches **and** PLC directory requests) so localhost targets work | + +## Handle resolution & DNS (wildcard requirement) + +Every bridged actor's atproto handle is a subdomain of `BRIDGE_HOSTNAME`: +communities get `technology.lemmy-world.`, users get +`alice.lemmy-world.` (dots replace the fediverse `!`/`@` +separators; dots inside the instance hostname become dashes; colliding +handles get a `-2`, `-3`, … suffix). + +For those handles to resolve, the operator **must configure wildcard DNS**: +`*.` → the bridge (a DNS wildcard matches multiple label +levels, so one record covers `*.lemmy-world.` and every +other bridged instance). The bridge then answers both resolution paths: + +- `GET /xrpc/com.atproto.identity.resolveHandle?handle=…` — the XRPC query; +- `GET https:///.well-known/atproto-did` — the HTTPS well-known + method; the wildcard DNS routes every bridged subdomain to the bridge, + which answers from the `Host` header. + +Note TLS: a single wildcard certificate only covers one label level, while +bridged handles sit two levels below `BRIDGE_HOSTNAME` — terminate TLS with +on-demand certificate issuance (e.g. Caddy) or per-instance wildcard certs. diff --git a/cmd/tidepool/main.go b/cmd/tidepool/main.go index 452bab5..b6a4fd4 100644 --- a/cmd/tidepool/main.go +++ b/cmd/tidepool/main.go @@ -19,6 +19,8 @@ import ( "tidepool/internal/config" "tidepool/internal/db" + "tidepool/internal/identity" + "tidepool/internal/store" ) const ( @@ -75,6 +77,14 @@ func run(logger *slog.Logger) error { _, _ = w.Write([]byte("ok")) }) + // Handle resolution for the bridged handle space (task 03). Bridged + // handles are subdomains of BRIDGE_HOSTNAME; wildcard DNS routes them + // all here (see README, "Handle resolution & DNS"). + actors := store.NewBridgedActors(database) + resolver := identity.NewStoreResolver(actors, cfg.BridgeHostname, cfg.BridgeServiceDID) + router.Get("/xrpc/com.atproto.identity.resolveHandle", identity.ResolveHandleHandler(resolver, logger)) + router.Get("/.well-known/atproto-did", identity.WellKnownDIDHandler(resolver, logger)) + // Later tasks register here: AP inbox + WebFinger (02/06), // com.atproto.sync.* + subscribeRepos (04), vote aggregates XRPC (07). diff --git a/docker-compose.dev.yml b/docker-compose.dev.yml index ef1a8e4..8f90f79 100644 --- a/docker-compose.dev.yml +++ b/docker-compose.dev.yml @@ -51,6 +51,85 @@ services: profiles: - test + # PLC directory postgres — backing store for the local did:plc directory. + # Port 5444 stays clear of both the Tidepool (5442/5443) and Coves + # (5434/5435/5436) stacks. + postgres-plc: + image: postgres:16 + container_name: tidepool-dev-postgres-plc + ports: + - "${POSTGRES_PLC_PORT:-5444}:5432" + environment: + POSTGRES_DB: plc_dev + POSTGRES_USER: plc_user + POSTGRES_PASSWORD: plc_password + volumes: + - postgres-plc-data:/var/lib/postgresql/data + networks: + - tidepool-dev + healthcheck: + test: ["CMD-SHELL", "pg_isready -U plc_user -d plc_dev"] + interval: 5s + timeout: 5s + retries: 5 + profiles: + - plc + + # Local PLC directory (the real did-method-plc server), modeled on the + # Coves dev stack. Minting tests run against this — NEVER against the + # public https://plc.directory. + # + # NOTE: this maps host port 3002, the same port the Coves dev PLC + # (coves-dev-plc) uses — run one or the other. The identity tests only + # need *something* answering at TIDEPOOL_TEST_PLC_URL (default + # http://localhost:3002), so an already-running Coves PLC works as-is. + # + # Usage: docker compose -f docker-compose.dev.yml --profile plc up -d + # First start clones and builds did-method-plc (several minutes); the + # volume caches it for later runs. + plc-directory: + image: node:18-alpine + container_name: tidepool-dev-plc + ports: + - "${PLC_PORT:-3002}:3000" + working_dir: /app + command: > + sh -c " + if [ ! -d '/app/.git' ]; then + echo 'First run: Installing PLC directory...' && + apk add --no-cache git python3 make g++ yarn && + git clone https://github.com/did-method-plc/did-method-plc.git . && + yarn install --frozen-lockfile && + yarn build && + echo 'PLC directory installed successfully!' + fi && + cd packages/server && + yarn start + " + environment: + DATABASE_URL: postgresql://plc_user:plc_password@postgres-plc:5432/plc_dev?sslmode=disable + DEBUG_MODE: "1" + LOG_ENABLED: "true" + LOG_LEVEL: debug + LOG_DESTINATION: "1" + NODE_ENV: development + PORT: 3000 + volumes: + - plc-app-data:/app + networks: + - tidepool-dev + depends_on: + postgres-plc: + condition: service_healthy + healthcheck: + test: ["CMD", "wget", "--spider", "-q", "http://localhost:3000/_health"] + interval: 10s + timeout: 5s + retries: 10 + start_period: 300s + profiles: + - plc + networks: tidepool-dev: driver: bridge @@ -61,3 +140,7 @@ volumes: name: tidepool-dev-postgres-data postgres-test-data: name: tidepool-test-postgres-data + postgres-plc-data: + name: tidepool-dev-postgres-plc-data + plc-app-data: + name: tidepool-dev-plc-app-data diff --git a/go.mod b/go.mod index 95235ce..de19ed9 100644 --- a/go.mod +++ b/go.mod @@ -5,7 +5,12 @@ go 1.25.7 require ( github.com/bluesky-social/indigo v0.0.0-20260202181658-ea3d39eec464 github.com/go-chi/chi/v5 v5.3.1 + github.com/ipfs/go-block-format v0.2.0 + github.com/ipfs/go-cid v0.4.1 + github.com/ipfs/go-ipld-format v0.6.0 + github.com/ipld/go-car v0.6.1-0.20230509095817-92d28eb23ba4 github.com/lib/pq v1.12.3 + github.com/multiformats/go-multihash v0.2.3 github.com/pressly/goose/v3 v3.27.2 github.com/stretchr/testify v1.11.1 golang.org/x/sync v0.21.0 @@ -13,10 +18,66 @@ require ( ) require ( + github.com/beorn7/perks v1.0.1 // indirect + github.com/cespare/xxhash/v2 v2.3.0 // indirect github.com/davecgh/go-spew v1.1.1 // indirect + github.com/earthboundkid/versioninfo/v2 v2.24.1 // indirect + github.com/go-logr/logr v1.4.3 // indirect + github.com/go-logr/stdr v1.2.2 // indirect + github.com/gogo/protobuf v1.3.2 // indirect + github.com/google/uuid v1.6.0 // indirect + github.com/hashicorp/golang-lru v1.0.2 // indirect + github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect + github.com/ipfs/bbloom v0.0.4 // indirect + github.com/ipfs/go-blockservice v0.5.2 // indirect + github.com/ipfs/go-datastore v0.6.0 // indirect + github.com/ipfs/go-ipfs-blockstore v1.3.1 // indirect + github.com/ipfs/go-ipfs-ds-help v1.1.1 // indirect + github.com/ipfs/go-ipfs-exchange-interface v0.2.1 // indirect + github.com/ipfs/go-ipfs-util v0.0.3 // indirect + github.com/ipfs/go-ipld-cbor v0.1.0 // indirect + github.com/ipfs/go-ipld-legacy v0.2.1 // indirect + github.com/ipfs/go-log v1.0.5 // indirect + github.com/ipfs/go-log/v2 v2.5.1 // indirect + github.com/ipfs/go-merkledag v0.11.0 // indirect + github.com/ipfs/go-metrics-interface v0.0.1 // indirect + github.com/ipfs/go-verifcid v0.0.3 // indirect + github.com/ipld/go-codec-dagpb v1.6.0 // indirect + github.com/ipld/go-ipld-prime v0.21.0 // indirect + github.com/jbenet/goprocess v0.1.4 // indirect + github.com/klauspost/cpuid/v2 v2.2.7 // indirect + github.com/mattn/go-isatty v0.0.21 // indirect + github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0 // indirect github.com/mfridman/interpolate v0.0.2 // indirect + github.com/minio/sha256-simd v1.0.1 // indirect + github.com/mr-tron/base58 v1.2.0 // indirect + github.com/multiformats/go-base32 v0.1.0 // indirect + github.com/multiformats/go-base36 v0.2.0 // indirect + github.com/multiformats/go-multibase v0.2.0 // indirect + github.com/multiformats/go-varint v0.0.7 // indirect + github.com/opentracing/opentracing-go v1.2.0 // indirect github.com/pmezard/go-difflib v1.0.0 // indirect + github.com/polydawn/refmt v0.89.1-0.20221221234430-40501e09de1f // indirect + github.com/prometheus/client_golang v1.17.0 // indirect + github.com/prometheus/client_model v0.5.0 // indirect + github.com/prometheus/common v0.45.0 // indirect + github.com/prometheus/procfs v0.20.1 // indirect github.com/sethvargo/go-retry v0.3.0 // indirect + github.com/spaolacci/murmur3 v1.1.0 // indirect + github.com/whyrusleeping/cbor-gen v0.2.1-0.20241030202151-b7a6831be65e // indirect + gitlab.com/yawning/secp256k1-voi v0.0.0-20230925100816-f2616030848b // indirect + gitlab.com/yawning/tuplehash v0.0.0-20230713102510-df83abbf9a02 // indirect + go.opentelemetry.io/auto/sdk v1.2.1 // indirect + go.opentelemetry.io/otel v1.43.0 // indirect + go.opentelemetry.io/otel/metric v1.43.0 // indirect + go.opentelemetry.io/otel/trace v1.43.0 // indirect + go.uber.org/atomic v1.11.0 // indirect go.uber.org/multierr v1.11.0 // indirect + go.uber.org/zap v1.26.0 // indirect + golang.org/x/crypto v0.52.0 // indirect + golang.org/x/sys v0.45.0 // indirect + golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 // indirect + google.golang.org/protobuf v1.36.11 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect + lukechampine.com/blake3 v1.2.1 // indirect ) diff --git a/go.sum b/go.sum index eace9f9..5b19034 100644 --- a/go.sum +++ b/go.sum @@ -1,43 +1,344 @@ +github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= +github.com/benbjohnson/clock v1.1.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA= +github.com/benbjohnson/clock v1.3.0 h1:ip6w0uFQkncKQ979AypyG0ER7mqUSBdKLOgAle/AT8A= +github.com/benbjohnson/clock v1.3.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA= +github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= +github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= github.com/bluesky-social/indigo v0.0.0-20260202181658-ea3d39eec464 h1:jL6cPOk1CZ8H06sEn+WFGWufHmqkawsGyDRl+BJhQjs= github.com/bluesky-social/indigo v0.0.0-20260202181658-ea3d39eec464/go.mod h1:VG/LeqLGNI3Ew7lsYixajnZGFfWPv144qbUddh+Oyag= +github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= +github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU= +github.com/cskr/pubsub v1.0.2 h1:vlOzMhl6PFn60gRlTQQsIfVwaPB/B/8MziK8FhEPt/0= +github.com/cskr/pubsub v1.0.2/go.mod h1:/8MzYXk/NJAz782G8RPkFzXTZVu63VotefPnR9TIRis= +github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/decred/dcrd/dcrec/secp256k1/v4 v4.2.0 h1:8UrgZ3GkP4i/CLijOJx79Yu+etlyjdBU4sfcs2WYQMs= +github.com/decred/dcrd/dcrec/secp256k1/v4 v4.2.0/go.mod h1:v57UDF4pDQJcEfFUCRop3lJL149eHGSe9Jvczhzjo/0= github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY= github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto= +github.com/earthboundkid/versioninfo/v2 v2.24.1 h1:SJTMHaoUx3GzjjnUO1QzP3ZXK6Ee/nbWyCm58eY3oUg= +github.com/earthboundkid/versioninfo/v2 v2.24.1/go.mod h1:VcWEooDEuyUJnMfbdTh0uFN4cfEIg+kHMuWB2CDCLjw= +github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8= +github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0= github.com/go-chi/chi/v5 v5.3.1 h1:3j4HZLGZQ3JpMCrPJF/Jl3mYJfWLKBfNJ6quurUGCf8= github.com/go-chi/chi/v5 v5.3.1/go.mod h1:R+tYY2hNuVUUjxoPtqUdgBqevM9s9njzkTLutVsOCto= +github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= +github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI= +github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY= +github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag= +github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE= +github.com/go-yaml/yaml v2.1.0+incompatible/go.mod h1:w2MrLa16VYP0jy6N7M5kHaCkaLENm+P+Tv+MfurjSw0= +github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q= +github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/google/gopacket v1.1.19 h1:ves8RnFZPGiFnTS0uPQStjwru6uO6h+nlr9j6fL7kF8= +github.com/google/gopacket v1.1.19/go.mod h1:iJ8V8n6KS+z2U1A8pUwu8bW5SyEMkXJB8Yo/Vo+TKTo= +github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI= github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1 h1:EGx4pi6eqNxGaHF6qqu48+N2wcFQ5qg5FXgOdqsJ5d8= +github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY= +github.com/hashicorp/golang-lru v1.0.2 h1:dV3g9Z/unq5DpblPpw+Oqcv4dU/1omnb4Ok8iPY6p1c= +github.com/hashicorp/golang-lru v1.0.2/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4= +github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k= +github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM= +github.com/huin/goupnp v1.0.3 h1:N8No57ls+MnjlB+JPiCVSOyy/ot7MJTqlo7rn+NYSqQ= +github.com/huin/goupnp v1.0.3/go.mod h1:ZxNlw5WqJj6wSsRK5+YfflQGXYfccj5VgQsMNixHM7Y= +github.com/ipfs/bbloom v0.0.4 h1:Gi+8EGJ2y5qiD5FbsbpX/TMNcJw8gSqr7eyjHa4Fhvs= +github.com/ipfs/bbloom v0.0.4/go.mod h1:cS9YprKXpoZ9lT0n/Mw/a6/aFV6DTjTLYHeA+gyqMG0= +github.com/ipfs/go-bitswap v0.11.0 h1:j1WVvhDX1yhG32NTC9xfxnqycqYIlhzEzLXG/cU1HyQ= +github.com/ipfs/go-bitswap v0.11.0/go.mod h1:05aE8H3XOU+LXpTedeAS0OZpcO1WFsj5niYQH9a1Tmk= +github.com/ipfs/go-block-format v0.2.0 h1:ZqrkxBA2ICbDRbK8KJs/u0O3dlp6gmAuuXUJNiW1Ycs= +github.com/ipfs/go-block-format v0.2.0/go.mod h1:+jpL11nFx5A/SPpsoBn6Bzkra/zaArfSmsknbPMYgzM= +github.com/ipfs/go-blockservice v0.5.2 h1:in9Bc+QcXwd1apOVM7Un9t8tixPKdaHQFdLSUM1Xgk8= +github.com/ipfs/go-blockservice v0.5.2/go.mod h1:VpMblFEqG67A/H2sHKAemeH9vlURVavlysbdUI632yk= +github.com/ipfs/go-cid v0.4.1 h1:A/T3qGvxi4kpKWWcPC/PgbvDA2bjVLO7n4UeVwnbs/s= +github.com/ipfs/go-cid v0.4.1/go.mod h1:uQHwDeX4c6CtyrFwdqyhpNcxVewur1M7l7fNU7LKwZk= +github.com/ipfs/go-datastore v0.6.0 h1:JKyz+Gvz1QEZw0LsX1IBn+JFCJQH4SJVFtM4uWU0Myk= +github.com/ipfs/go-datastore v0.6.0/go.mod h1:rt5M3nNbSO/8q1t4LNkLyUwRs8HupMeN/8O4Vn9YAT8= +github.com/ipfs/go-detect-race v0.0.1 h1:qX/xay2W3E4Q1U7d9lNs1sU9nvguX0a7319XbyQ6cOk= +github.com/ipfs/go-detect-race v0.0.1/go.mod h1:8BNT7shDZPo99Q74BpGMK+4D8Mn4j46UU0LZ723meps= +github.com/ipfs/go-ipfs-blockstore v1.3.1 h1:cEI9ci7V0sRNivqaOr0elDsamxXFxJMMMy7PTTDQNsQ= +github.com/ipfs/go-ipfs-blockstore v1.3.1/go.mod h1:KgtZyc9fq+P2xJUiCAzbRdhhqJHvsw8u2Dlqy2MyRTE= +github.com/ipfs/go-ipfs-blocksutil v0.0.1 h1:Eh/H4pc1hsvhzsQoMEP3Bke/aW5P5rVM1IWFJMcGIPQ= +github.com/ipfs/go-ipfs-blocksutil v0.0.1/go.mod h1:Yq4M86uIOmxmGPUHv/uI7uKqZNtLb449gwKqXjIsnRk= +github.com/ipfs/go-ipfs-delay v0.0.1 h1:r/UXYyRcddO6thwOnhiznIAiSvxMECGgtv35Xs1IeRQ= +github.com/ipfs/go-ipfs-delay v0.0.1/go.mod h1:8SP1YXK1M1kXuc4KJZINY3TQQ03J2rwBG9QfXmbRPrw= +github.com/ipfs/go-ipfs-ds-help v1.1.1 h1:B5UJOH52IbcfS56+Ul+sv8jnIV10lbjLF5eOO0C66Nw= +github.com/ipfs/go-ipfs-ds-help v1.1.1/go.mod h1:75vrVCkSdSFidJscs8n4W+77AtTpCIAdDGAwjitJMIo= +github.com/ipfs/go-ipfs-exchange-interface v0.2.1 h1:jMzo2VhLKSHbVe+mHNzYgs95n0+t0Q69GQ5WhRDZV/s= +github.com/ipfs/go-ipfs-exchange-interface v0.2.1/go.mod h1:MUsYn6rKbG6CTtsDp+lKJPmVt3ZrCViNyH3rfPGsZ2E= +github.com/ipfs/go-ipfs-exchange-offline v0.3.0 h1:c/Dg8GDPzixGd0MC8Jh6mjOwU57uYokgWRFidfvEkuA= +github.com/ipfs/go-ipfs-exchange-offline v0.3.0/go.mod h1:MOdJ9DChbb5u37M1IcbrRB02e++Z7521fMxqCNRrz9s= +github.com/ipfs/go-ipfs-pq v0.0.2 h1:e1vOOW6MuOwG2lqxcLA+wEn93i/9laCY8sXAw76jFOY= +github.com/ipfs/go-ipfs-pq v0.0.2/go.mod h1:LWIqQpqfRG3fNc5XsnIhz/wQ2XXGyugQwls7BgUmUfY= +github.com/ipfs/go-ipfs-routing v0.3.0 h1:9W/W3N+g+y4ZDeffSgqhgo7BsBSJwPMcyssET9OWevc= +github.com/ipfs/go-ipfs-routing v0.3.0/go.mod h1:dKqtTFIql7e1zYsEuWLyuOU+E0WJWW8JjbTPLParDWo= +github.com/ipfs/go-ipfs-util v0.0.3 h1:2RFdGez6bu2ZlZdI+rWfIdbQb1KudQp3VGwPtdNCmE0= +github.com/ipfs/go-ipfs-util v0.0.3/go.mod h1:LHzG1a0Ig4G+iZ26UUOMjHd+lfM84LZCrn17xAKWBvs= +github.com/ipfs/go-ipld-cbor v0.1.0 h1:dx0nS0kILVivGhfWuB6dUpMa/LAwElHPw1yOGYopoYs= +github.com/ipfs/go-ipld-cbor v0.1.0/go.mod h1:U2aYlmVrJr2wsUBU67K4KgepApSZddGRDWBYR0H4sCk= +github.com/ipfs/go-ipld-format v0.6.0 h1:VEJlA2kQ3LqFSIm5Vu6eIlSxD/Ze90xtc4Meten1F5U= +github.com/ipfs/go-ipld-format v0.6.0/go.mod h1:g4QVMTn3marU3qXchwjpKPKgJv+zF+OlaKMyhJ4LHPg= +github.com/ipfs/go-ipld-legacy v0.2.1 h1:mDFtrBpmU7b//LzLSypVrXsD8QxkEWxu5qVxN99/+tk= +github.com/ipfs/go-ipld-legacy v0.2.1/go.mod h1:782MOUghNzMO2DER0FlBR94mllfdCJCkTtDtPM51otM= +github.com/ipfs/go-log v1.0.5 h1:2dOuUCB1Z7uoczMWgAyDck5JLb72zHzrMnGnCNNbvY8= +github.com/ipfs/go-log v1.0.5/go.mod h1:j0b8ZoR+7+R99LD9jZ6+AJsrzkPbSXbZfGakb5JPtIo= +github.com/ipfs/go-log/v2 v2.1.3/go.mod h1:/8d0SH3Su5Ooc31QlL1WysJhvyOTDCjcCZ9Axpmri6g= +github.com/ipfs/go-log/v2 v2.5.1 h1:1XdUzF7048prq4aBjDQQ4SL5RxftpRGdXhNRwKSAlcY= +github.com/ipfs/go-log/v2 v2.5.1/go.mod h1:prSpmC1Gpllc9UYWxDiZDreBYw7zp4Iqp1kOLU9U5UI= +github.com/ipfs/go-merkledag v0.11.0 h1:DgzwK5hprESOzS4O1t/wi6JDpyVQdvm9Bs59N/jqfBY= +github.com/ipfs/go-merkledag v0.11.0/go.mod h1:Q4f/1ezvBiJV0YCIXvt51W/9/kqJGH4I1LsA7+djsM4= +github.com/ipfs/go-metrics-interface v0.0.1 h1:j+cpbjYvu4R8zbleSs36gvB7jR+wsL2fGD6n0jO4kdg= +github.com/ipfs/go-metrics-interface v0.0.1/go.mod h1:6s6euYU4zowdslK0GKHmqaIZ3j/b/tL7HTWtJ4VPgWY= +github.com/ipfs/go-peertaskqueue v0.8.0 h1:JyNO144tfu9bx6Hpo119zvbEL9iQ760FHOiJYsUjqaU= +github.com/ipfs/go-peertaskqueue v0.8.0/go.mod h1:cz8hEnnARq4Du5TGqiWKgMr/BOSQ5XOgMOh1K5YYKKM= +github.com/ipfs/go-verifcid v0.0.3 h1:gmRKccqhWDocCRkC+a59g5QW7uJw5bpX9HWBevXa0zs= +github.com/ipfs/go-verifcid v0.0.3/go.mod h1:gcCtGniVzelKrbk9ooUSX/pM3xlH73fZZJDzQJRvOUw= +github.com/ipld/go-car v0.6.1-0.20230509095817-92d28eb23ba4 h1:oFo19cBmcP0Cmg3XXbrr0V/c+xU9U1huEZp8+OgBzdI= +github.com/ipld/go-car v0.6.1-0.20230509095817-92d28eb23ba4/go.mod h1:6nkFF8OmR5wLKBzRKi7/YFJpyYR7+oEn1DX+mMWnlLA= +github.com/ipld/go-codec-dagpb v1.6.0 h1:9nYazfyu9B1p3NAgfVdpRco3Fs2nFC72DqVsMj6rOcc= +github.com/ipld/go-codec-dagpb v1.6.0/go.mod h1:ANzFhfP2uMJxRBr8CE+WQWs5UsNa0pYtmKZ+agnUw9s= +github.com/ipld/go-ipld-prime v0.21.0 h1:n4JmcpOlPDIxBcY037SVfpd1G+Sj1nKZah0m6QH9C2E= +github.com/ipld/go-ipld-prime v0.21.0/go.mod h1:3RLqy//ERg/y5oShXXdx5YIp50cFGOanyMctpPjsvxQ= +github.com/jackpal/go-nat-pmp v1.0.2 h1:KzKSgb7qkJvOUTqYl9/Hg/me3pWgBmERKrTGD7BdWus= +github.com/jackpal/go-nat-pmp v1.0.2/go.mod h1:QPH045xvCAeXUZOxsnwmrtiCoxIr9eob+4orBN1SBKc= +github.com/jbenet/go-cienv v0.1.0/go.mod h1:TqNnHUmJgXau0nCzC7kXWeotg3J9W34CUv5Djy1+FlA= +github.com/jbenet/goprocess v0.1.4 h1:DRGOFReOMqqDNXwW70QkacFW0YN9QnwLV0Vqk+3oU0o= +github.com/jbenet/goprocess v0.1.4/go.mod h1:5yspPrukOVuOLORacaBi858NqyClJPQxYZlqdZVfqY4= +github.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7C0MuV77Wo= +github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU= +github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= +github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= +github.com/klauspost/cpuid/v2 v2.2.7 h1:ZWSB3igEs+d0qvnxR/ZBzXVmxkgt8DdzP6m9pfuVLDM= +github.com/klauspost/cpuid/v2 v2.2.7/go.mod h1:Lcz8mBdAVJIBVzewtcLocK12l3Y+JytZYpaMropDUws= +github.com/koron/go-ssdp v0.0.3 h1:JivLMY45N76b4p/vsWGOKewBQu6uf39y8l+AQ7sDKx8= +github.com/koron/go-ssdp v0.0.3/go.mod h1:b2MxI6yh02pKrsyNoQUsk4+YNikaGhe4894J+Q5lDvA= +github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= +github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= +github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= +github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= +github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= github.com/lib/pq v1.12.3 h1:tTWxr2YLKwIvK90ZXEw8GP7UFHtcbTtty8zsI+YjrfQ= github.com/lib/pq v1.12.3/go.mod h1:/p+8NSbOcwzAEI7wiMXFlgydTwcgTr3OSKMsD2BitpA= +github.com/libp2p/go-buffer-pool v0.1.0 h1:oK4mSFcQz7cTQIfqbe4MIj9gLW+mnanjyFtc6cdF0Y8= +github.com/libp2p/go-buffer-pool v0.1.0/go.mod h1:N+vh8gMqimBzdKkSMVuydVDq+UV5QTWy5HSiZacSbPg= +github.com/libp2p/go-cidranger v1.1.0 h1:ewPN8EZ0dd1LSnrtuwd4709PXVcITVeuwbag38yPW7c= +github.com/libp2p/go-cidranger v1.1.0/go.mod h1:KWZTfSr+r9qEo9OkI9/SIEeAtw+NNoU0dXIXt15Okic= +github.com/libp2p/go-libp2p v0.22.0 h1:2Tce0kHOp5zASFKJbNzRElvh0iZwdtG5uZheNW8chIw= +github.com/libp2p/go-libp2p v0.22.0/go.mod h1:UDolmweypBSjQb2f7xutPnwZ/fxioLbMBxSjRksxxU4= +github.com/libp2p/go-libp2p-asn-util v0.2.0 h1:rg3+Os8jbnO5DxkC7K/Utdi+DkY3q/d1/1q+8WeNAsw= +github.com/libp2p/go-libp2p-asn-util v0.2.0/go.mod h1:WoaWxbHKBymSN41hWSq/lGKJEca7TNm58+gGJi2WsLI= +github.com/libp2p/go-libp2p-record v0.2.0 h1:oiNUOCWno2BFuxt3my4i1frNrt7PerzB3queqa1NkQ0= +github.com/libp2p/go-libp2p-record v0.2.0/go.mod h1:I+3zMkvvg5m2OcSdoL0KPljyJyvNDFGKX7QdlpYUcwk= +github.com/libp2p/go-libp2p-testing v0.12.0 h1:EPvBb4kKMWO29qP4mZGyhVzUyR25dvfUIK5WDu6iPUA= +github.com/libp2p/go-libp2p-testing v0.12.0/go.mod h1:KcGDRXyN7sQCllucn1cOOS+Dmm7ujhfEyXQL5lvkcPg= +github.com/libp2p/go-msgio v0.2.0 h1:W6shmB+FeynDrUVl2dgFQvzfBZcXiyqY4VmpQLu9FqU= +github.com/libp2p/go-msgio v0.2.0/go.mod h1:dBVM1gW3Jk9XqHkU4eKdGvVHdLa51hoGfll6jMJMSlY= +github.com/libp2p/go-nat v0.1.0 h1:MfVsH6DLcpa04Xr+p8hmVRG4juse0s3J8HyNWYHffXg= +github.com/libp2p/go-nat v0.1.0/go.mod h1:X7teVkwRHNInVNWQiO/tAiAVRwSr5zoRz4YSTC3uRBM= +github.com/libp2p/go-netroute v0.2.0 h1:0FpsbsvuSnAhXFnCY0VLFbJOzaK0VnP0r1QT/o4nWRE= +github.com/libp2p/go-netroute v0.2.0/go.mod h1:Vio7LTzZ+6hoT4CMZi5/6CpY3Snzh2vgZhWgxMNwlQI= +github.com/libp2p/go-openssl v0.1.0 h1:LBkKEcUv6vtZIQLVTegAil8jbNpJErQ9AnT+bWV+Ooo= +github.com/libp2p/go-openssl v0.1.0/go.mod h1:OiOxwPpL3n4xlenjx2h7AwSGaFSC/KZvf6gNdOBQMtc= +github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94= github.com/mattn/go-isatty v0.0.21 h1:xYae+lCNBP7QuW4PUnNG61ffM4hVIfm+zUzDuSzYLGs= github.com/mattn/go-isatty v0.0.21/go.mod h1:ZXfXG4SQHsB/w3ZeOYbR0PrPwLy+n6xiMrJlRFqopa4= +github.com/mattn/go-pointer v0.0.1 h1:n+XhsuGeVO6MEAp7xyEukFINEa+Quek5psIR/ylA6o0= +github.com/mattn/go-pointer v0.0.1/go.mod h1:2zXcozF6qYGgmsG+SeTZz3oAbFLdD3OWqnUbNvJZAlc= +github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0 h1:jWpvCLoY8Z/e3VKvlsiIGKtc+UG6U5vzxaoagmhXfyg= +github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0/go.mod h1:QUyp042oQthUoa9bqDv0ER0wrtXnBruoNd7aNjkbP+k= github.com/mfridman/interpolate v0.0.2 h1:pnuTK7MQIxxFz1Gr+rjSIx9u7qVjf5VOoM/u6BbAxPY= github.com/mfridman/interpolate v0.0.2/go.mod h1:p+7uk6oE07mpE/Ik1b8EckO0O4ZXiGAfshKBWLUM9Xg= +github.com/miekg/dns v1.1.50 h1:DQUfb9uc6smULcREF09Uc+/Gd46YWqJd5DbpPE9xkcA= +github.com/miekg/dns v1.1.50/go.mod h1:e3IlAVfNqAllflbibAZEWOXOQ+Ynzk/dDozDxY7XnME= +github.com/minio/sha256-simd v1.0.1 h1:6kaan5IFmwTNynnKKpDHe6FWHohJOHhCPchzK49dzMM= +github.com/minio/sha256-simd v1.0.1/go.mod h1:Pz6AKMiUdngCLpeTL/RJY1M9rUuPMYujV5xJjtbRSN8= +github.com/mr-tron/base58 v1.2.0 h1:T/HDJBh4ZCPbU39/+c3rRvE0uKBQlU27+QI8LJ4t64o= +github.com/mr-tron/base58 v1.2.0/go.mod h1:BinMc/sQntlIE1frQmRFPUoPA1Zkr8VRgBdjWI2mNwc= +github.com/multiformats/go-base32 v0.1.0 h1:pVx9xoSPqEIQG8o+UbAe7DNi51oej1NtK+aGkbLYxPE= +github.com/multiformats/go-base32 v0.1.0/go.mod h1:Kj3tFY6zNr+ABYMqeUNeGvkIC/UYgtWibDcT0rExnbI= +github.com/multiformats/go-base36 v0.2.0 h1:lFsAbNOGeKtuKozrtBsAkSVhv1p9D0/qedU9rQyccr0= +github.com/multiformats/go-base36 v0.2.0/go.mod h1:qvnKE++v+2MWCfePClUEjE78Z7P2a1UV0xHgWc0hkp4= +github.com/multiformats/go-multiaddr v0.7.0 h1:gskHcdaCyPtp9XskVwtvEeQOG465sCohbQIirSyqxrc= +github.com/multiformats/go-multiaddr v0.7.0/go.mod h1:Fs50eBDWvZu+l3/9S6xAE7ZYj6yhxlvaVZjakWN7xRs= +github.com/multiformats/go-multiaddr-dns v0.3.1 h1:QgQgR+LQVt3NPTjbrLLpsaT2ufAA2y0Mkk+QRVJbW3A= +github.com/multiformats/go-multiaddr-dns v0.3.1/go.mod h1:G/245BRQ6FJGmryJCrOuTdB37AMA5AMOVuO6NY3JwTk= +github.com/multiformats/go-multiaddr-fmt v0.1.0 h1:WLEFClPycPkp4fnIzoFoV9FVd49/eQsuaL3/CWe167E= +github.com/multiformats/go-multiaddr-fmt v0.1.0/go.mod h1:hGtDIW4PU4BqJ50gW2quDuPVjyWNZxToGUh/HwTZYJo= +github.com/multiformats/go-multibase v0.2.0 h1:isdYCVLvksgWlMW9OZRYJEa9pZETFivncJHmHnnd87g= +github.com/multiformats/go-multibase v0.2.0/go.mod h1:bFBZX4lKCA/2lyOFSAoKH5SS6oPyjtnzK/XTFDPkNuk= +github.com/multiformats/go-multicodec v0.9.0 h1:pb/dlPnzee/Sxv/j4PmkDRxCOi3hXTz3IbPKOXWJkmg= +github.com/multiformats/go-multicodec v0.9.0/go.mod h1:L3QTQvMIaVBkXOXXtVmYE+LI16i14xuaojr/H7Ai54k= +github.com/multiformats/go-multihash v0.2.3 h1:7Lyc8XfX/IY2jWb/gI7JP+o7JEq9hOa7BFvVU9RSh+U= +github.com/multiformats/go-multihash v0.2.3/go.mod h1:dXgKXCXjBzdscBLk9JkjINiEsCKRVch90MdaGiKsvSM= +github.com/multiformats/go-multistream v0.3.3 h1:d5PZpjwRgVlbwfdTDjife7XszfZd8KYWfROYFlGcR8o= +github.com/multiformats/go-multistream v0.3.3/go.mod h1:ODRoqamLUsETKS9BNcII4gcRsJBU5VAwRIv7O39cEXg= +github.com/multiformats/go-varint v0.0.7 h1:sWSGR+f/eu5ABZA2ZpYKBILXTTs9JWpdEM/nEGOHFS8= +github.com/multiformats/go-varint v0.0.7/go.mod h1:r8PUYw/fD/SjBCiKOoDlGF6QawOELpZAu9eioSos/OU= github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w= github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls= +github.com/opentracing/opentracing-go v1.2.0 h1:uEJPy/1a5RIPAJ0Ov+OIO8OxWu77jEv+1B0VhjKrZUs= +github.com/opentracing/opentracing-go v1.2.0/go.mod h1:GxEUsuufX4nBwe+T+Wl9TAgYrxe9dPLANfrWvHYVTgc= +github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/polydawn/refmt v0.89.1-0.20221221234430-40501e09de1f h1:VXTQfuJj9vKR4TCkEuWIckKvdHFeJH/huIFJ9/cXOB0= +github.com/polydawn/refmt v0.89.1-0.20221221234430-40501e09de1f/go.mod h1:/zvteZs/GwLtCgZ4BL6CBsk9IKIlexP43ObX9AxTqTw= github.com/pressly/goose/v3 v3.27.2 h1:FjKNzcmMdGrQlSIu5alMSmakQtJFBgtw+A0bb1p/LC8= github.com/pressly/goose/v3 v3.27.2/go.mod h1:qWW+/8dkVtJYjJrbIpwD5xxnEJTUKvxkQ9JKQp9LaIM= +github.com/prometheus/client_golang v1.17.0 h1:rl2sfwZMtSthVU752MqfjQozy7blglC+1SOtjMAMh+Q= +github.com/prometheus/client_golang v1.17.0/go.mod h1:VeL+gMmOAxkS2IqfCq0ZmHSL+LjWfWDUmp1mBz9JgUY= +github.com/prometheus/client_model v0.5.0 h1:VQw1hfvPvk3Uv6Qf29VrPF32JB6rtbgI6cYPYQjL0Qw= +github.com/prometheus/client_model v0.5.0/go.mod h1:dTiFglRmd66nLR9Pv9f0mZi7B7fk5Pm3gvsjB5tr+kI= +github.com/prometheus/common v0.45.0 h1:2BGz0eBc2hdMDLnO/8n0jeB3oPrt2D08CekT0lneoxM= +github.com/prometheus/common v0.45.0/go.mod h1:YJmSTw9BoKxJplESWWxlbyttQR4uaEcGyv9MZjVOJsY= +github.com/prometheus/procfs v0.20.1 h1:XwbrGOIplXW/AU3YhIhLODXMJYyC1isLFfYCsTEycfc= +github.com/prometheus/procfs v0.20.1/go.mod h1:o9EMBZGRyvDrSPH1RqdxhojkuXstoe4UlK79eF5TGGo= github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE= github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo= +github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= +github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ= +github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc= +github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= github.com/sethvargo/go-retry v0.3.0 h1:EEt31A35QhrcRZtrYFDTBg91cqZVnFL2navjDrah2SE= github.com/sethvargo/go-retry v0.3.0/go.mod h1:mNX17F0C/HguQMyMyJxcnU471gOZGxCLyYaFyAZraas= +github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc= +github.com/smartystreets/assertions v1.2.0 h1:42S6lae5dvLc7BrLu/0ugRtcFVjoJNMC/N3yZFZkDFs= +github.com/smartystreets/assertions v1.2.0/go.mod h1:tcbTF8ujkAEcZ8TElKY+i30BzYlVhC/LOxJk7iOWnoo= +github.com/smartystreets/goconvey v1.7.2 h1:9RBaZCeXEQ3UselpuwUQHltGVXvdwm6cv1hgR6gDIPg= +github.com/smartystreets/goconvey v1.7.2/go.mod h1:Vw0tHAZW6lzCRk3xgdin6fKYcG+G3Pg9vgXWeJpQFMM= +github.com/spacemonkeygo/spacelog v0.0.0-20180420211403-2296661a0572 h1:RC6RW7j+1+HkWaX/Yh71Ee5ZHaHYt7ZP4sQgUrm6cDU= +github.com/spacemonkeygo/spacelog v0.0.0-20180420211403-2296661a0572/go.mod h1:w0SWMsp6j9O/dk4/ZpIhL+3CkG8ofA2vuv7k+ltqUMc= +github.com/spaolacci/murmur3 v1.1.0 h1:7c1g84S4BPRrfL5Xrdp6fOJ206sU9y293DDHaoy0bLI= +github.com/spaolacci/murmur3 v1.1.0/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= +github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= +github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= +github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= +github.com/urfave/cli v1.22.10/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0= +github.com/warpfork/go-testmark v0.12.1 h1:rMgCpJfwy1sJ50x0M0NgyphxYYPMOODIJHhsXyEHU0s= +github.com/warpfork/go-testmark v0.12.1/go.mod h1:kHwy7wfvGSPh1rQJYKayD4AbtNaeyZdcGi9tNJTaa5Y= +github.com/warpfork/go-wish v0.0.0-20220906213052-39a1cc7a02d0 h1:GDDkbFiaK8jsSDJfjId/PEGEShv6ugrt4kYsC5UIDaQ= +github.com/warpfork/go-wish v0.0.0-20220906213052-39a1cc7a02d0/go.mod h1:x6AKhvSSexNrVSrViXSHUEbICjmGXhtgABaHIySUSGw= +github.com/whyrusleeping/cbor-gen v0.2.1-0.20241030202151-b7a6831be65e h1:28X54ciEwwUxyHn9yrZfl5ojgF4CBNLWX7LR0rvBkf4= +github.com/whyrusleeping/cbor-gen v0.2.1-0.20241030202151-b7a6831be65e/go.mod h1:pM99HXyEbSQHcosHc0iW7YFmwnscr+t9Te4ibko05so= +github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= +github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= +github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k= +gitlab.com/yawning/secp256k1-voi v0.0.0-20230925100816-f2616030848b h1:CzigHMRySiX3drau9C6Q5CAbNIApmLdat5jPMqChvDA= +gitlab.com/yawning/secp256k1-voi v0.0.0-20230925100816-f2616030848b/go.mod h1:/y/V339mxv2sZmYYR64O07VuCpdNZqCTwO8ZcouTMI8= +gitlab.com/yawning/tuplehash v0.0.0-20230713102510-df83abbf9a02 h1:qwDnMxjkyLmAFgcfgTnfJrmYKWhHnci3GjDqcZp1M3Q= +gitlab.com/yawning/tuplehash v0.0.0-20230713102510-df83abbf9a02/go.mod h1:JTnUj0mpYiAsuZLmKjTx/ex3AtMowcCgnE7YNyCEP0I= +go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64= +go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y= +go.opentelemetry.io/otel v1.43.0 h1:mYIM03dnh5zfN7HautFE4ieIig9amkNANT+xcVxAj9I= +go.opentelemetry.io/otel v1.43.0/go.mod h1:JuG+u74mvjvcm8vj8pI5XiHy1zDeoCS2LB1spIq7Ay0= +go.opentelemetry.io/otel/metric v1.43.0 h1:d7638QeInOnuwOONPp4JAOGfbCEpYb+K6DVWvdxGzgM= +go.opentelemetry.io/otel/metric v1.43.0/go.mod h1:RDnPtIxvqlgO8GRW18W6Z/4P462ldprJtfxHxyKd2PY= +go.opentelemetry.io/otel/trace v1.43.0 h1:BkNrHpup+4k4w+ZZ86CZoHHEkohws8AY+WTX09nk+3A= +go.opentelemetry.io/otel/trace v1.43.0/go.mod h1:/QJhyVBUUswCphDVxq+8mld+AvhXZLhe+8WVFxiFff0= +go.uber.org/atomic v1.6.0/go.mod h1:sABNBOSYdrvTF6hTgEIbc7YasKWGhgEQZyfxyTvoXHQ= +go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= +go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE= +go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0= +go.uber.org/goleak v1.1.11-0.20210813005559-691160354723/go.mod h1:cwTWslyiVhfpKIDGSZEM2HlOvcqm+tG4zioyIeLoqMQ= +go.uber.org/goleak v1.2.0 h1:xqgm/S+aQvhWFTtR0XK3Jvg7z8kGV8P4X14IzwN3Eqk= +go.uber.org/goleak v1.2.0/go.mod h1:XJYK+MuIchqpmGmUSAzotztawfKvYLUIgg7guXrwVUo= +go.uber.org/multierr v1.5.0/go.mod h1:FeouvMocqHpRaaGuG9EjoKcStLC43Zu/fmqdUMPcKYU= +go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU= go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0= go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y= +go.uber.org/tools v0.0.0-20190618225709-2cfd321de3ee/go.mod h1:vJERXedbb3MVM5f9Ejo0C68/HhF8uaILCdgjnY+goOA= +go.uber.org/zap v1.16.0/go.mod h1:MA8QOfq0BHJwdXa996Y4dYkAqRKB8/1K1QMMZVaNZjQ= +go.uber.org/zap v1.19.1/go.mod h1:j3DNczoxDZroyBnOT1L/Q79cfUMGZxlv/9dzN7SM1rI= +go.uber.org/zap v1.26.0 h1:sI7k6L95XOKS281NhVKOFCUNIvv9e0w4BF8N3u+tCRo= +go.uber.org/zap v1.26.0/go.mod h1:dtElttAiwGvoJ/vj4IwHBS/gXsEu/pZ50mUIRWuG0so= +golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= +golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= +golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= +golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= +golang.org/x/crypto v0.52.0 h1:RMs7fP2rXdep0CftQlK8Uf+kibLm7qkCcradZWYz988= +golang.org/x/crypto v0.52.0/go.mod h1:1QgfPxDqh0T2M/elOJtp9RvuR95kVjir0e6/BvEmGbc= +golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= +golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc= +golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= +golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= +golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= +golang.org/x/mod v0.14.0 h1:dGoOF9QVLYng8IHTm7BAyWqCqSheQ5pYWGhzW00YJr0= +golang.org/x/mod v0.14.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c= +golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= +golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= +golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= +golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM= +golang.org/x/net v0.55.0 h1:bcvxaJn3e1U6InsFWt1JUq1aSjnRxLzT2rtD2KfkDF8= +golang.org/x/net v0.55.0/go.mod h1:L5U2KuzuOe1lY7Z+aWVIKK6qEeJXnXV9yzGA+WCHJww= +golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM= golang.org/x/sync v0.21.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= +golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.45.0 h1:dO4czNzziLiiXplLQgBCEpCvXQ3dnkn0SdaZSYdQ+FY= golang.org/x/sys v0.45.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= +golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= +golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= +golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U= golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno= -gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= +golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= +golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= +golang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= +golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= +golang.org/x/tools v0.0.0-20191029041327-9cc4af7d6b2c/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191029190741-b9c20aec41a5/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= +golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= +golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= +golang.org/x/tools v0.15.0 h1:zdAyfUGbYmuVokhzVmghFl2ZJh5QhcfebBgmVPFYA+8= +golang.org/x/tools v0.15.0/go.mod h1:hpksKq4dtpQWS1uQ61JkdqWM3LscIS6Slf+VVkm+wQk= +golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 h1:+cNy6SZtPcJQH3LJVLOSmiC7MMxXNOb3PU/VUEz+EhU= +golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028/go.mod h1:NDW/Ps6MPRej6fsCIbMTohpP40sJ/P/vI1MoTEGwX90= +google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE= +google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= +gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI= +gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= +gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= +gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg= +lukechampine.com/blake3 v1.2.1 h1:YuqqRuaqsGV71BV/nm9xlI0MKUv4QC54jQnBChWbGnI= +lukechampine.com/blake3 v1.2.1/go.mod h1:0OFRp7fBtAylGVCO40o87sbupkyIGgbpv1+M1k1LM6k= modernc.org/libc v1.73.4 h1:+ra4Ui8ngyt8HDcO1FTDPWlkAh6yOdaO2yAoh8MddQA= modernc.org/libc v1.73.4/go.mod h1:DXZ3eO8qMCNn2SnmTNCiC71nJ9Rcq3PsnpU6Vc4rWK8= modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU= diff --git a/internal/ap/egress.go b/internal/ap/egress.go index 28470e8..92988e8 100644 --- a/internal/ap/egress.go +++ b/internal/ap/egress.go @@ -4,11 +4,30 @@ import ( "context" "fmt" "net" + "net/http" "net/url" + "time" "tidepool/internal/errors" ) +// NewGuardedHTTPClient returns an *http.Client whose transport enforces the +// same SSRF egress guard the AP client uses (resolved-IP validation at dial +// time). Non-AP outbound HTTP — the identity package's PLC directory client +// — uses this so every egress path in the bridge shares one guard. As with +// the AP client, allowPrivate must only be true in development/tests +// (config.AllowPrivateAddresses, env ALLOW_PRIVATE_FETCH). +func NewGuardedHTTPClient(allowPrivate bool, timeout time.Duration) *http.Client { + if timeout == 0 { + timeout = DefaultRequestTimeout + } + guard := newEgressGuard(allowPrivate) + return &http.Client{ + Timeout: timeout, + Transport: guardedTransport(nil, guard), + } +} + // egressGuard blocks outbound requests to addresses an SSRF attacker would // pivot through: loopback, RFC1918 private, link-local, unique-local, // multicast, unspecified, and the cloud-metadata endpoint. It rejects diff --git a/internal/config/config.go b/internal/config/config.go index 0d6abaf..02f7f7f 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -4,6 +4,8 @@ package config import ( + "encoding/base64" + "encoding/hex" "fmt" "log/slog" "os" @@ -28,9 +30,20 @@ type Config struct { // e.g. "tidepool.example". Used for WebFinger, actor IDs, and handles. BridgeHostname string // PLCDirectoryURL is the did:plc directory used to mint and resolve DIDs. + // The dev default is a LOCAL directory (docker compose `plc` profile); + // production points at https://plc.directory. Nothing ever falls back to + // the live directory implicitly. PLCDirectoryURL string + // BridgeKEK is the 32-byte key-encryption key that seals per-actor + // signing keys (and the escrow rotation key) at rest, AES-256-GCM. + // Set BRIDGE_KEK to 64 hex chars or standard base64 of 32 bytes. The + // development default is a fixed, publicly known key — never usable in + // production, where BRIDGE_KEK is required. + BridgeKEK []byte // BridgeServiceDID optionally pins a pre-provisioned service DID for the - // bridge's own actor. Empty means task 03 will mint one on first run. + // bridge's own actor. Service-DID bootstrap is deferred: task 06 wires + // the service actor; until then an empty value is handled gracefully + // (the bridge hostname simply does not resolve to a DID). BridgeServiceDID string // UserAgent is sent on all outbound HTTP requests (signed fetches etc.). UserAgent string @@ -78,6 +91,19 @@ func Load(logger *slog.Logger) (*Config, error) { return nil, err } + // The dev-default KEK is fixed and public (sha256 of a known string): + // fine for local development, catastrophic in production, hence the + // required-in-production rule shared with every other stringVar. + kekEncoded, err := stringVar(logger, isDevelopment, "BRIDGE_KEK", + "9a80812a2a5e298fe6b36ba6ba99f33ca42a7a5b1cae7ff43a4b338bbbdd6a34") + if err != nil { + return nil, err + } + cfg.BridgeKEK, err = decodeKEK(kekEncoded) + if err != nil { + return nil, err + } + // Optional in every environment: an operator may pre-provision the // bridge's service DID, otherwise identity bootstrap mints one. cfg.BridgeServiceDID = os.Getenv("BRIDGE_SERVICE_DID") @@ -106,6 +132,26 @@ func (c *Config) IsDevelopment() bool { return c.Environment == EnvironmentDevelopment } +// decodeKEK parses the BRIDGE_KEK value: 64 hex chars or standard base64, +// either way decoding to exactly 32 bytes. +func decodeKEK(encoded string) ([]byte, error) { + encoded = strings.TrimSpace(encoded) + if raw, err := hex.DecodeString(encoded); err == nil { + if len(raw) != 32 { + return nil, fmt.Errorf("config: BRIDGE_KEK must decode to 32 bytes, got %d", len(raw)) + } + return raw, nil + } + raw, err := base64.StdEncoding.DecodeString(encoded) + if err != nil { + return nil, fmt.Errorf("config: BRIDGE_KEK must be 64 hex chars or base64 of 32 bytes: %w", err) + } + if len(raw) != 32 { + return nil, fmt.Errorf("config: BRIDGE_KEK must decode to 32 bytes, got %d", len(raw)) + } + return raw, nil +} + // boolVar reports whether an environment variable is set to a truthy value // ("1", "true", "yes", case-insensitive). func boolVar(name string) bool { diff --git a/internal/config/config_test.go b/internal/config/config_test.go index b445580..79df715 100644 --- a/internal/config/config_test.go +++ b/internal/config/config_test.go @@ -17,7 +17,7 @@ func clearConfigEnv(t *testing.T) { t.Helper() for _, name := range []string{ "ENVIRONMENT", "DATABASE_URL", "LISTEN_ADDR", "BRIDGE_HOSTNAME", - "PLC_DIRECTORY_URL", "BRIDGE_SERVICE_DID", "USER_AGENT", + "PLC_DIRECTORY_URL", "BRIDGE_SERVICE_DID", "USER_AGENT", "BRIDGE_KEK", } { t.Setenv(name, "") } @@ -34,9 +34,11 @@ func TestLoad_DevelopmentDefaults(t *testing.T) { assert.Equal(t, "postgres://tidepool:tidepool@localhost:5442/tidepool_dev?sslmode=disable", cfg.DatabaseURL) assert.Equal(t, ":8091", cfg.ListenAddr) assert.Equal(t, "localhost", cfg.BridgeHostname) - assert.Equal(t, "http://localhost:3002", cfg.PLCDirectoryURL) + assert.Equal(t, "http://localhost:3002", cfg.PLCDirectoryURL, + "the dev default PLC directory must be LOCAL, never the live plc.directory") assert.Empty(t, cfg.BridgeServiceDID, "service DID is optional") assert.Equal(t, "tidepool/0.1 (+https://localhost)", cfg.UserAgent) + assert.Len(t, cfg.BridgeKEK, 32, "dev-default KEK decodes to 32 bytes") } func TestLoad_ExplicitValuesWin(t *testing.T) { @@ -75,12 +77,36 @@ func TestLoad_ProductionWithAllValues(t *testing.T) { t.Setenv("LISTEN_ADDR", ":8080") t.Setenv("BRIDGE_HOSTNAME", "tidepool.example") t.Setenv("PLC_DIRECTORY_URL", "https://plc.directory") + t.Setenv("BRIDGE_KEK", "sfDrM4bIeCJp01ZBTArLPJXNQlD7pcYFsod2An6UAF0=") // base64 form cfg, err := Load(discardLogger()) require.NoError(t, err) assert.False(t, cfg.IsDevelopment()) assert.Equal(t, "tidepool/0.1 (+https://tidepool.example)", cfg.UserAgent, "user agent default derives from the bridge hostname") + assert.Len(t, cfg.BridgeKEK, 32, "base64 KEK decodes to 32 bytes") +} + +func TestLoad_ProductionRequiresKEK(t *testing.T) { + clearConfigEnv(t) + t.Setenv("ENVIRONMENT", EnvironmentProduction) + t.Setenv("DATABASE_URL", "postgres://prod/db") + t.Setenv("LISTEN_ADDR", ":8080") + t.Setenv("BRIDGE_HOSTNAME", "tidepool.example") + t.Setenv("PLC_DIRECTORY_URL", "https://plc.directory") + + _, err := Load(discardLogger()) + require.Error(t, err, "production must never run on the public dev-default KEK") + assert.Contains(t, err.Error(), "BRIDGE_KEK") +} + +func TestLoad_RejectsBadKEK(t *testing.T) { + clearConfigEnv(t) + t.Setenv("BRIDGE_KEK", "too-short") + + _, err := Load(discardLogger()) + require.Error(t, err) + assert.Contains(t, err.Error(), "BRIDGE_KEK") } func TestLoad_RejectsUnknownEnvironment(t *testing.T) { diff --git a/internal/db/migrations/006_create_repo_tables.sql b/internal/db/migrations/006_create_repo_tables.sql new file mode 100644 index 0000000..a6ee40b --- /dev/null +++ b/internal/db/migrations/006_create_repo_tables.sql @@ -0,0 +1,65 @@ +-- +goose Up +-- The virtual-PDS storage spine (task 03): every bridged DID gets a real +-- signed MST repo whose blocks and head pointer live here, plus a durable +-- firehose event log that task 04 serves over com.atproto.sync.subscribeRepos. + +-- blocks holds every IPLD block (records, MST nodes, commits) for every +-- bridged repo, keyed per-DID so repos stay independently exportable. +-- Blocks are content-addressed and therefore immutable; superseded MST +-- nodes are kept (cheap, and historical CAR slices in firehose_events +-- reference them). Garbage collection of unreachable blocks is a later +-- optimization, deliberately not v1. +CREATE TABLE blocks ( + did TEXT NOT NULL CHECK (did <> ''), + cid TEXT NOT NULL CHECK (cid <> ''), + bytes BYTEA NOT NULL, + created_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP, + CONSTRAINT blocks_pkey PRIMARY KEY (did, cid) +); + +-- repo_state is the mutable head pointer per repo: the latest signed commit +-- CID and its rev TID. Cross-process serialization comes from a GLOBAL +-- advisory transaction lock every commit takes before reading this table +-- (internal/repo commitAdvisoryLockKey) — NOT from the row lock: SELECT ... +-- FOR UPDATE on a missing row locks nothing, so the row lock alone cannot +-- make genesis commits race-free. The row lock is kept only as a backstop. +CREATE TABLE repo_state ( + did TEXT PRIMARY KEY CHECK (did <> ''), + head_cid TEXT NOT NULL CHECK (head_cid <> ''), + rev TEXT NOT NULL CHECK (rev <> ''), + updated_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP +); + +-- firehose_events is the durable subscribeRepos backlog: one row per commit, +-- appended in the SAME transaction as the commit itself so the stream can +-- never miss a write. seq is the firehose cursor: because every commit +-- transaction holds the global advisory lock until it commits, seq order +-- equals commit-visibility order, so a tailer doing `WHERE seq > cursor` +-- never permanently skips an event. car holds the CAR slice for the +-- #commit message (root = commit CID, written first; contains the commit +-- block plus the MST and record blocks new in this commit). ops is the +-- JSON op list ({action, path, cid, prev}); prev_data_cid is the MST root +-- before this commit (sync v1.1 prevData, NULL on a repo's genesis commit); +-- since_rev is the previous commit's rev (the #commit frame's `since` +-- field, NULL on genesis) — persisted because task 04 cannot reconstruct +-- it once older events are pruned. +CREATE TABLE firehose_events ( + seq BIGSERIAL PRIMARY KEY, + did TEXT NOT NULL CHECK (did <> ''), + commit_cid TEXT NOT NULL CHECK (commit_cid <> ''), + prev_data_cid TEXT, + since_rev TEXT, + rev TEXT NOT NULL CHECK (rev <> ''), + ops JSONB NOT NULL, + car BYTEA NOT NULL, + created_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP +); + +-- Task 04 serves per-DID catch-up (getRepo diff-since) and global cursor +-- scans; both want these indexes. +CREATE INDEX idx_firehose_events_did_seq ON firehose_events (did, seq); + +-- +goose Down +DROP TABLE IF EXISTS firehose_events; +DROP TABLE IF EXISTS repo_state; +DROP TABLE IF EXISTS blocks; diff --git a/internal/identity/e2e_test.go b/internal/identity/e2e_test.go new file mode 100644 index 0000000..8e658c9 --- /dev/null +++ b/internal/identity/e2e_test.go @@ -0,0 +1,165 @@ +package identity + +import ( + "bytes" + "testing" + + indigorepo "github.com/bluesky-social/indigo/atproto/repo" + + "github.com/bluesky-social/indigo/atproto/atcrypto" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "tidepool/internal/errors" + "tidepool/internal/repo" + "tidepool/internal/store" + "tidepool/internal/testutil" +) + +// TestMintedIdentityEndToEnd is the task-03 definition of done, verbatim: +// mint a DID on a local PLC directory, create its repo, write a profile +// record, read it back, and verify the commit signature with the minted +// key (the exact key registered as the DID's verification key). +func TestMintedIdentityEndToEnd(t *testing.T) { + database := testutil.DB(t) + testutil.Truncate(t, database, + "bridged_actors", "blocks", "repo_state", "firehose_events") + actors := store.NewBridgedActors(database) + minter, custodian, _, _ := testMinter(t, actors) // skips without local PLC + ctx := t.Context() + + // Mint. + identity, err := minter.MintActor(ctx, MintRequest{ + ActorType: store.ActorTypeGroup, + PreferredUsername: "technology", + Instance: "lemmy.world", + }) + require.NoError(t, err) + + // Persist the bridged actor the way tasks 05/06 will. + _, err = actors.UpsertActor(ctx, store.BridgedActor{ + APActorID: "https://lemmy.world/c/technology", + ActorType: store.ActorTypeGroup, + DID: identity.DID, + Handle: identity.Handle, + SigningKeyEncrypted: identity.SigningKeyEncrypted, + ConsentState: store.ConsentStateOK, + }) + require.NoError(t, err) + + // Create the repo and write the profile record (rkey "self", per the + // materialization principle: profiles land before any content). + manager, err := repo.NewManager(database, NewActorKeys(actors, custodian), nil) + require.NoError(t, err) + profile := map[string]any{ + "$type": "social.coves.community.profile", + "name": "technology", + "description": "bridged from lemmy.world by Tidepool", + } + res, err := manager.PutRecord(ctx, identity.DID, "social.coves.community.profile", "self", profile) + require.NoError(t, err) + require.NotEmpty(t, res.Rev) + + // Read it back. + got, gotCID, err := manager.GetRecord(ctx, identity.DID, "social.coves.community.profile", "self") + require.NoError(t, err) + assert.Equal(t, res.RecordCID, gotCID) + assert.Equal(t, "technology", got["name"]) + + // Export and verify the commit signature with the minted key — parsed + // from the did:key the PLC operation registered, not from local state. + carBytes, err := manager.ExportCAR(ctx, identity.DID) + require.NoError(t, err) + commit, _, err := indigorepo.LoadRepoFromCAR(ctx, bytes.NewReader(carBytes)) + require.NoError(t, err) + assert.Equal(t, identity.DID, commit.DID) + + mintedPub, err := atcrypto.ParsePublicDIDKey(identity.DIDKey) + require.NoError(t, err) + require.NoError(t, commit.VerifySignature(mintedPub), + "commit signature must verify with the key minted into the DID document") +} + +// TestTombstoneFreezesRepoAtCommitLayer wires a real repo.Manager to +// identity.ActorKeys (the production SigningKeys implementation) and proves +// the consent-revocation freeze end to end: tombstoning an actor blocks new +// writes at the commit layer while keeping their existing records deletable +// (task 05's Delete(Actor) → scrub-records flow). PLC-independent: the actor +// is created directly in the store with a fake DID. +func TestTombstoneFreezesRepoAtCommitLayer(t *testing.T) { + database := testutil.DB(t) + testutil.Truncate(t, database, + "bridged_actors", "blocks", "repo_state", "firehose_events") + actors := store.NewBridgedActors(database) + custodian := testCustodian(t) + ctx := t.Context() + + signing, err := atcrypto.GeneratePrivateKeyK256() + require.NoError(t, err) + const ( + did = "did:plc:ewvi7nxzyoun6zhxrhs64oiz" + apActorID = "https://lemmy.world/u/alice" + collection = "social.coves.community.post" + rkey = "3mks4zznhkard" // any valid TID-form rkey + ) + sealed, err := custodian.EncryptActorKey(did, signing) + require.NoError(t, err) + _, err = actors.UpsertActor(ctx, store.BridgedActor{ + APActorID: apActorID, + ActorType: store.ActorTypePerson, + DID: did, + Handle: "alice.lemmy-world.tidepool.example", + SigningKeyEncrypted: sealed, + ConsentState: store.ConsentStateOK, + }) + require.NoError(t, err) + + manager, err := repo.NewManager(database, NewActorKeys(actors, custodian), nil) + require.NoError(t, err) + + record := map[string]any{"$type": collection, "text": "hello"} + before, err := manager.PutRecord(ctx, did, collection, rkey, record) + require.NoError(t, err) + require.False(t, before.NoOp) + + firehoseCount := func() int { + var n int + require.NoError(t, database.QueryRowContext(ctx, + `SELECT COUNT(*) FROM firehose_events WHERE did = $1`, did).Scan(&n)) + return n + } + require.Equal(t, 1, firehoseCount()) + + // Consent flip: the actor tombstones (terminal). + require.NoError(t, actors.SetConsentState(ctx, apActorID, store.ConsentStateDeleted)) + + // (a) New writes are frozen at the commit layer, surfacing the consent + // state — not a generic failure. + _, err = manager.PutRecord(ctx, did, collection, "3mks522bgpxc5", + map[string]any{"$type": collection, "text": "after tombstone"}) + require.Error(t, err) + assert.True(t, errors.IsTombstoned(err), + "post-tombstone writes must fail with IsTombstoned") + assert.False(t, errors.IsNotFound(err), "tombstoned must not read as missing") + + // (b) The head did not move. + headCID, headRev, err := manager.Head(ctx, did) + require.NoError(t, err) + assert.Equal(t, before.CommitCID, headCID, "blocked write must not move the head") + assert.Equal(t, before.Rev, headRev, "blocked write must not advance the rev") + + // (c) No firehose event leaked from the blocked write. + assert.Equal(t, 1, firehoseCount(), "blocked write must not emit a firehose event") + + // (d) Deleting the existing record still works (KeyUseDelete releases + // the key) and emits a firehose event: scrubbing IS the consent intent. + deleted, err := manager.DeleteRecord(ctx, did, collection, rkey) + require.NoError(t, err, + "tombstoned actor's records must remain deletable (scrub flow)") + assert.Greater(t, deleted.Rev, before.Rev) + assert.Greater(t, deleted.Seq, before.Seq, "the delete must emit a firehose event") + assert.Equal(t, 2, firehoseCount()) + + _, _, err = manager.GetRecord(ctx, did, collection, rkey) + assert.True(t, errors.IsNotFound(err), "the scrubbed record must be gone") +} diff --git a/internal/identity/handles.go b/internal/identity/handles.go new file mode 100644 index 0000000..65842fd --- /dev/null +++ b/internal/identity/handles.go @@ -0,0 +1,141 @@ +package identity + +import ( + "context" + "encoding/json" + "log/slog" + "net" + "net/http" + "strings" + + comatproto "github.com/bluesky-social/indigo/api/atproto" + + "tidepool/internal/errors" + "tidepool/internal/store" +) + +// Handle resolution for the bridge's handle space. +// +// Every bridged handle is a subdomain of BRIDGE_HOSTNAME +// (alice.lemmy-world.), so the operator must run wildcard DNS +// pointing *. (which in DNS matches multiple label levels) +// at the bridge, with TLS termination covering those names. With that in +// place both atproto resolution paths work against this package: +// +// - com.atproto.identity.resolveHandle XRPC (ResolveHandleHandler), +// - the HTTPS well-known method: GET https:///.well-known/atproto-did +// lands here because of the wildcard DNS, and WellKnownDIDHandler answers +// from the Host header. +// +// The DNS requirement is documented in the README. + +// Resolver answers "which DID owns this bridged handle". Implemented by +// StoreResolver; an interface so the HTTP handlers are testable without +// postgres. +type Resolver interface { + // ResolveHandle returns the DID for a handle. Unknown handles — and + // handles of tombstoned actors, whose bridged identity is frozen — are + // errors satisfying errors.IsNotFound. + ResolveHandle(ctx context.Context, handle string) (string, error) +} + +// StoreResolver resolves bridged handles from the bridged_actors table, +// plus the bridge's own service DID for the bare bridge hostname. +type StoreResolver struct { + actors store.BridgedActors + // bridgeHostname (lowercased) resolves to serviceDID when non-empty: + // the bridge's own service identity (config.BridgeServiceDID). + bridgeHostname string + serviceDID string +} + +// NewStoreResolver builds the store-backed resolver. serviceDID may be +// empty (the bridge's own handle then does not resolve). +func NewStoreResolver(actors store.BridgedActors, bridgeHostname, serviceDID string) *StoreResolver { + return &StoreResolver{ + actors: actors, + bridgeHostname: strings.ToLower(bridgeHostname), + serviceDID: serviceDID, + } +} + +func (r *StoreResolver) ResolveHandle(ctx context.Context, handle string) (string, error) { + handle = strings.ToLower(strings.TrimSuffix(handle, ".")) + if handle == "" { + return "", errors.NewValidationError("handle", "must not be empty") + } + if r.serviceDID != "" && handle == r.bridgeHostname { + return r.serviceDID, nil + } + actor, err := r.actors.GetByHandle(ctx, handle) + if err != nil { + return "", err + } + // A tombstoned actor's identity is frozen (task-01 semantics): its + // handle stops resolving rather than advertising a dead repo. + if actor.ConsentState == store.ConsentStateDeleted { + return "", errors.NewNotFoundError("handle", handle) + } + return actor.DID, nil +} + +// ResolveHandleHandler implements the com.atproto.identity.resolveHandle +// XRPC query: GET /xrpc/com.atproto.identity.resolveHandle?handle= +// → {"did": "..."}. Unresolvable handles return the standard XRPC error +// shape with status 400, matching the reference implementation. +func ResolveHandleHandler(resolver Resolver, logger *slog.Logger) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + handle := strings.TrimPrefix(r.URL.Query().Get("handle"), "@") + if handle == "" { + writeXRPCError(w, http.StatusBadRequest, "InvalidRequest", "missing required parameter: handle") + return + } + did, err := resolver.ResolveHandle(r.Context(), handle) + switch { + case err == nil: + writeJSON(w, http.StatusOK, comatproto.IdentityResolveHandle_Output{Did: did}) + case errors.IsNotFound(err): + writeXRPCError(w, http.StatusBadRequest, "HandleNotFound", "Unable to resolve handle") + case errors.IsValidation(err): + writeXRPCError(w, http.StatusBadRequest, "InvalidRequest", "invalid handle") + default: + logger.Error("resolveHandle failed", "handle", handle, "error", err) + writeXRPCError(w, http.StatusInternalServerError, "InternalServerError", "internal error") + } + } +} + +// WellKnownDIDHandler serves GET /.well-known/atproto-did, treating the +// request's Host as the handle being verified (that is how the HTTPS +// resolution method addresses a handle; wildcard DNS routes every bridged +// subdomain here). Responds text/plain with the bare DID. +func WellKnownDIDHandler(resolver Resolver, logger *slog.Logger) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + host := r.Host + if h, _, err := net.SplitHostPort(host); err == nil { + host = h + } + did, err := resolver.ResolveHandle(r.Context(), host) + switch { + case err == nil: + w.Header().Set("Content-Type", "text/plain; charset=utf-8") + w.WriteHeader(http.StatusOK) + _, _ = w.Write([]byte(did)) + case errors.IsNotFound(err) || errors.IsValidation(err): + http.Error(w, "no atproto DID for this host", http.StatusNotFound) + default: + logger.Error("well-known atproto-did lookup failed", "host", host, "error", err) + http.Error(w, "internal error", http.StatusInternalServerError) + } + } +} + +func writeJSON(w http.ResponseWriter, status int, body any) { + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(status) + _ = json.NewEncoder(w).Encode(body) +} + +func writeXRPCError(w http.ResponseWriter, status int, code, message string) { + writeJSON(w, status, map[string]string{"error": code, "message": message}) +} diff --git a/internal/identity/handles_test.go b/internal/identity/handles_test.go new file mode 100644 index 0000000..d2ccdf3 --- /dev/null +++ b/internal/identity/handles_test.go @@ -0,0 +1,178 @@ +package identity + +import ( + "context" + "encoding/json" + "log/slog" + "net/http" + "net/http/httptest" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "tidepool/internal/errors" + "tidepool/internal/store" + "tidepool/internal/testutil" +) + +// fakeResolver backs the HTTP handler unit tests. +type fakeResolver struct { + handles map[string]string + err error +} + +func (f *fakeResolver) ResolveHandle(_ context.Context, handle string) (string, error) { + if f.err != nil { + return "", f.err + } + if did, ok := f.handles[handle]; ok { + return did, nil + } + return "", errors.NewNotFoundError("handle", handle) +} + +func testLogger() *slog.Logger { return slog.Default() } + +func TestResolveHandleHandler(t *testing.T) { + resolver := &fakeResolver{handles: map[string]string{ + "alice.lemmy-world.tidepool.example": "did:plc:ewvi7nxzyoun6zhxrhs64oiz", + }} + handler := ResolveHandleHandler(resolver, testLogger()) + + t.Run("resolves known handle", func(t *testing.T) { + rec := httptest.NewRecorder() + handler(rec, httptest.NewRequest(http.MethodGet, + "/xrpc/com.atproto.identity.resolveHandle?handle=alice.lemmy-world.tidepool.example", nil)) + require.Equal(t, http.StatusOK, rec.Code) + var out struct { + Did string `json:"did"` + } + require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &out)) + assert.Equal(t, "did:plc:ewvi7nxzyoun6zhxrhs64oiz", out.Did) + }) + + t.Run("strips leading @", func(t *testing.T) { + rec := httptest.NewRecorder() + handler(rec, httptest.NewRequest(http.MethodGet, + "/xrpc/com.atproto.identity.resolveHandle?handle=%40alice.lemmy-world.tidepool.example", nil)) + assert.Equal(t, http.StatusOK, rec.Code) + }) + + t.Run("unknown handle is 400 HandleNotFound", func(t *testing.T) { + rec := httptest.NewRecorder() + handler(rec, httptest.NewRequest(http.MethodGet, + "/xrpc/com.atproto.identity.resolveHandle?handle=nobody.example.com", nil)) + require.Equal(t, http.StatusBadRequest, rec.Code) + var out struct { + Error string `json:"error"` + } + require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &out)) + assert.Equal(t, "HandleNotFound", out.Error) + }) + + t.Run("missing parameter is 400 InvalidRequest", func(t *testing.T) { + rec := httptest.NewRecorder() + handler(rec, httptest.NewRequest(http.MethodGet, + "/xrpc/com.atproto.identity.resolveHandle", nil)) + require.Equal(t, http.StatusBadRequest, rec.Code) + var out struct { + Error string `json:"error"` + } + require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &out)) + assert.Equal(t, "InvalidRequest", out.Error) + }) + + t.Run("internal error is 500", func(t *testing.T) { + broken := ResolveHandleHandler(&fakeResolver{err: context.DeadlineExceeded}, testLogger()) + rec := httptest.NewRecorder() + broken(rec, httptest.NewRequest(http.MethodGet, + "/xrpc/com.atproto.identity.resolveHandle?handle=x.example.com", nil)) + assert.Equal(t, http.StatusInternalServerError, rec.Code) + }) +} + +func TestWellKnownDIDHandler(t *testing.T) { + resolver := &fakeResolver{handles: map[string]string{ + "alice.lemmy-world.tidepool.example": "did:plc:ewvi7nxzyoun6zhxrhs64oiz", + }} + handler := WellKnownDIDHandler(resolver, testLogger()) + + t.Run("serves DID for known host", func(t *testing.T) { + req := httptest.NewRequest(http.MethodGet, "/.well-known/atproto-did", nil) + req.Host = "alice.lemmy-world.tidepool.example" + rec := httptest.NewRecorder() + handler(rec, req) + require.Equal(t, http.StatusOK, rec.Code) + assert.Equal(t, "did:plc:ewvi7nxzyoun6zhxrhs64oiz", rec.Body.String()) + }) + + t.Run("strips port from host", func(t *testing.T) { + req := httptest.NewRequest(http.MethodGet, "/.well-known/atproto-did", nil) + req.Host = "alice.lemmy-world.tidepool.example:8091" + rec := httptest.NewRecorder() + handler(rec, req) + require.Equal(t, http.StatusOK, rec.Code) + assert.Equal(t, "did:plc:ewvi7nxzyoun6zhxrhs64oiz", rec.Body.String(), + "the port-stripped host must resolve to the handle's DID") + }) + + t.Run("unknown host is 404", func(t *testing.T) { + req := httptest.NewRequest(http.MethodGet, "/.well-known/atproto-did", nil) + req.Host = "unknown.tidepool.example" + rec := httptest.NewRecorder() + handler(rec, req) + assert.Equal(t, http.StatusNotFound, rec.Code) + }) +} + +func TestStoreResolver(t *testing.T) { + database := testutil.DB(t) + testutil.Truncate(t, database, "bridged_actors") + actors := store.NewBridgedActors(database) + resolver := NewStoreResolver(actors, "tidepool.example", "did:plc:44ybard66vv44zksje25o7dz") + ctx := t.Context() + + const ( + did = "did:plc:ewvi7nxzyoun6zhxrhs64oiz" + handle = "alice.lemmy-world.tidepool.example" + ) + _, err := actors.UpsertActor(ctx, store.BridgedActor{ + APActorID: "https://lemmy.world/u/alice", + ActorType: store.ActorTypePerson, + DID: did, + Handle: handle, + ConsentState: store.ConsentStateOK, + }) + require.NoError(t, err) + + t.Run("resolves bridged handle", func(t *testing.T) { + got, err := resolver.ResolveHandle(ctx, handle) + require.NoError(t, err) + assert.Equal(t, did, got) + }) + + t.Run("case-insensitive with trailing dot", func(t *testing.T) { + got, err := resolver.ResolveHandle(ctx, "Alice.Lemmy-World.Tidepool.Example.") + require.NoError(t, err) + assert.Equal(t, did, got) + }) + + t.Run("bridge hostname resolves to service DID", func(t *testing.T) { + got, err := resolver.ResolveHandle(ctx, "tidepool.example") + require.NoError(t, err) + assert.Equal(t, "did:plc:44ybard66vv44zksje25o7dz", got) + }) + + t.Run("unknown handle not found", func(t *testing.T) { + _, err := resolver.ResolveHandle(ctx, "nobody.lemmy-world.tidepool.example") + assert.True(t, errors.IsNotFound(err)) + }) + + t.Run("tombstoned actor stops resolving", func(t *testing.T) { + require.NoError(t, actors.SetConsentState(ctx, "https://lemmy.world/u/alice", store.ConsentStateDeleted)) + _, err := resolver.ResolveHandle(ctx, handle) + assert.True(t, errors.IsNotFound(err), + "a tombstoned actor's handle must not advertise its frozen repo") + }) +} diff --git a/internal/identity/keys.go b/internal/identity/keys.go new file mode 100644 index 0000000..64aad1c --- /dev/null +++ b/internal/identity/keys.go @@ -0,0 +1,208 @@ +// Package identity mints and custodies the atproto identities of bridged +// fediverse actors: did:plc creation against a PLC directory, per-actor +// secp256k1 signing keys held in escrow (AES-GCM encrypted at rest), and +// handle resolution for the bridge's subdomain handle space. +package identity + +import ( + "context" + "crypto/aes" + "crypto/cipher" + "crypto/rand" + "fmt" + + "github.com/bluesky-social/indigo/atproto/atcrypto" + + "tidepool/internal/errors" + "tidepool/internal/repo" + "tidepool/internal/store" +) + +// KEKSize is the required byte length of the bridge key-encryption key +// (BRIDGE_KEK): 32 bytes for AES-256-GCM. +const KEKSize = 32 + +// ciphertextVersion prefixes every sealed key so the format can evolve +// (e.g. KEK rotation with key IDs) without a schema change. +const ciphertextVersion byte = 1 + +// RotationKeyName is the service_keys row holding the bridge's escrow +// rotation key (encrypted with the KEK, unlike the AP-side RSA key that +// task 02 documented as stored in the clear — the rotation key controls +// every bridged DID, so it gets the stronger treatment). +const RotationKeyName = "plc-rotation" + +// aadContext strings bind each ciphertext to its purpose and owner so a +// sealed key copied into another row (or another column) fails to open. +const ( + actorKeyAADPrefix = "tidepool:actor-signing-key:v1:" + rotationKeyAAD = "tidepool:plc-rotation-key:v1" +) + +// Custodian seals and opens per-actor secp256k1 private keys with the +// bridge KEK (AES-256-GCM). Key claiming/migration is out of scope for v1, +// but the storage design allows it later: each actor's key is independent, +// bound to its DID via AAD, and exportable by decrypting and handing the +// key material to the user during a future claim flow. +type Custodian struct { + // aead is built once at construction and reused: DecryptActorKey sits + // on every commit's hot path, and cipher.AEAD is safe for concurrent + // use. + aead cipher.AEAD +} + +// NewCustodian validates the KEK and returns a Custodian. +func NewCustodian(kek []byte) (*Custodian, error) { + if len(kek) != KEKSize { + return nil, errors.NewValidationError("bridge_kek", + fmt.Sprintf("must be %d bytes, got %d", KEKSize, len(kek))) + } + block, err := aes.NewCipher(kek) + if err != nil { + return nil, fmt.Errorf("identity: init AES: %w", err) + } + gcm, err := cipher.NewGCM(block) + if err != nil { + return nil, fmt.Errorf("identity: init GCM: %w", err) + } + return &Custodian{aead: gcm}, nil +} + +// EncryptActorKey seals an actor's signing key for storage in +// bridged_actors.signing_key. The ciphertext is bound to the actor's DID: +// decrypting it under any other DID fails authentication. +func (c *Custodian) EncryptActorKey(did string, key *atcrypto.PrivateKeyK256) ([]byte, error) { + if did == "" { + return nil, errors.NewValidationError("did", "must not be empty") + } + return c.seal(key.Bytes(), []byte(actorKeyAADPrefix+did)) +} + +// DecryptActorKey opens a sealed actor signing key from +// bridged_actors.signing_key. did must be the DID the key was sealed for. +func (c *Custodian) DecryptActorKey(did string, ciphertext []byte) (*atcrypto.PrivateKeyK256, error) { + raw, err := c.open(ciphertext, []byte(actorKeyAADPrefix+did)) + if err != nil { + return nil, fmt.Errorf("identity: decrypt signing key for %s: %w", did, err) + } + key, err := atcrypto.ParsePrivateBytesK256(raw) + if err != nil { + return nil, fmt.Errorf("identity: parse signing key for %s: %w", did, err) + } + return key, nil +} + +// seal encrypts plaintext with AES-256-GCM under the KEK. Layout: +// version byte || 12-byte random nonce || GCM ciphertext+tag. +func (c *Custodian) seal(plaintext, aad []byte) ([]byte, error) { + nonce := make([]byte, c.aead.NonceSize()) + if _, err := rand.Read(nonce); err != nil { + return nil, fmt.Errorf("identity: generate nonce: %w", err) + } + out := make([]byte, 0, 1+len(nonce)+len(plaintext)+c.aead.Overhead()) + out = append(out, ciphertextVersion) + out = append(out, nonce...) + return c.aead.Seal(out, nonce, plaintext, aad), nil +} + +// open reverses seal. +func (c *Custodian) open(ciphertext, aad []byte) ([]byte, error) { + if len(ciphertext) < 1+c.aead.NonceSize()+c.aead.Overhead() { + return nil, errors.NewValidationError("ciphertext", "too short to be a sealed key") + } + if ciphertext[0] != ciphertextVersion { + return nil, errors.NewValidationError("ciphertext", + fmt.Sprintf("unknown ciphertext version %d", ciphertext[0])) + } + nonce := ciphertext[1 : 1+c.aead.NonceSize()] + sealed := ciphertext[1+c.aead.NonceSize():] + plaintext, err := c.aead.Open(nil, nonce, sealed, aad) + if err != nil { + return nil, fmt.Errorf("identity: open sealed key: %w", err) + } + return plaintext, nil +} + +// LoadOrCreateRotationKey returns the bridge's escrow rotation key, +// generating and persisting it (sealed with the KEK) on first run. The +// service_keys create-once semantics make the bootstrap race safe: a loser +// re-reads the winner's key. +func LoadOrCreateRotationKey(ctx context.Context, keys store.ServiceKeys, custodian *Custodian) (*atcrypto.PrivateKeyK256, error) { + stored, err := keys.Get(ctx, RotationKeyName) + if err == nil { + return decryptRotationKey(custodian, stored.PrivateKeyPEM) + } + if !errors.IsNotFound(err) { + return nil, fmt.Errorf("identity: load rotation key: %w", err) + } + + fresh, err := atcrypto.GeneratePrivateKeyK256() + if err != nil { + return nil, fmt.Errorf("identity: generate rotation key: %w", err) + } + sealed, err := custodian.seal(fresh.Bytes(), []byte(rotationKeyAAD)) + if err != nil { + return nil, fmt.Errorf("identity: seal rotation key: %w", err) + } + // NOTE: the column is named private_key_pem for the RSA service-actor + // key; for the rotation key it holds the sealed ciphertext instead. + if _, err := keys.Create(ctx, RotationKeyName, sealed); err != nil { + if errors.IsAlreadyExists(err) { + // Lost the bootstrap race: use the winner's key. + winner, getErr := keys.Get(ctx, RotationKeyName) + if getErr != nil { + return nil, fmt.Errorf("identity: reload rotation key after race: %w", getErr) + } + return decryptRotationKey(custodian, winner.PrivateKeyPEM) + } + return nil, fmt.Errorf("identity: persist rotation key: %w", err) + } + return fresh, nil +} + +func decryptRotationKey(custodian *Custodian, sealed []byte) (*atcrypto.PrivateKeyK256, error) { + raw, err := custodian.open(sealed, []byte(rotationKeyAAD)) + if err != nil { + return nil, fmt.Errorf("identity: decrypt rotation key: %w", err) + } + key, err := atcrypto.ParsePrivateBytesK256(raw) + if err != nil { + return nil, fmt.Errorf("identity: parse rotation key: %w", err) + } + return key, nil +} + +// ActorKeys resolves the signing key for a bridged DID, for the repo layer +// to sign commits with. It implements repo.SigningKeys. +type ActorKeys struct { + actors store.BridgedActors + custodian *Custodian +} + +// NewActorKeys builds the store-backed signing-key resolver. +func NewActorKeys(actors store.BridgedActors, custodian *Custodian) *ActorKeys { + return &ActorKeys{actors: actors, custodian: custodian} +} + +// SigningKey returns the decrypted signing key for a bridged DID. The +// consent gate is per KeyUse: a tombstoned actor (consent_state=deleted, +// terminal) never gets a key for repo.KeyUseWrite — that freeze is how +// deleted repos are prevented from growing — but repo.KeyUseDelete still +// releases the key, because scrubbing a deleted actor's records must always +// be possible (that IS the consent intent; task 05's Delete(Actor) → +// scrub-records flow depends on it). The write-freeze error satisfies +// errors.IsTombstoned; an actor without an escrowed key satisfies +// errors.IsNotFound. +func (a *ActorKeys) SigningKey(ctx context.Context, did string, use repo.KeyUse) (atcrypto.PrivateKey, error) { + actor, err := a.actors.GetByDID(ctx, did) + if err != nil { + return nil, err + } + if actor.ConsentState == store.ConsentStateDeleted && use != repo.KeyUseDelete { + return nil, errors.NewTombstonedError("bridged_actor", did) + } + if len(actor.SigningKeyEncrypted) == 0 { + return nil, errors.NewNotFoundError("signing_key", did) + } + return a.custodian.DecryptActorKey(did, actor.SigningKeyEncrypted) +} diff --git a/internal/identity/keys_test.go b/internal/identity/keys_test.go new file mode 100644 index 0000000..5c73bac --- /dev/null +++ b/internal/identity/keys_test.go @@ -0,0 +1,199 @@ +package identity + +import ( + "bytes" + "crypto/sha256" + "testing" + + "github.com/bluesky-social/indigo/atproto/atcrypto" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "tidepool/internal/errors" + "tidepool/internal/repo" + "tidepool/internal/store" + "tidepool/internal/testutil" +) + +func testKEK() []byte { + sum := sha256.Sum256([]byte("tidepool-test-kek")) + return sum[:] +} + +func testCustodian(t *testing.T) *Custodian { + t.Helper() + c, err := NewCustodian(testKEK()) + require.NoError(t, err) + return c +} + +func TestNewCustodian_RejectsBadKEKLength(t *testing.T) { + _, err := NewCustodian([]byte("short")) + require.Error(t, err) + assert.True(t, errors.IsValidation(err)) +} + +func TestCustodian_ActorKeyRoundTrip(t *testing.T) { + custodian := testCustodian(t) + key, err := atcrypto.GeneratePrivateKeyK256() + require.NoError(t, err) + + const did = "did:plc:ewvi7nxzyoun6zhxrhs64oiz" + sealed, err := custodian.EncryptActorKey(did, key) + require.NoError(t, err) + assert.NotContains(t, string(sealed), string(key.Bytes()), + "ciphertext must not contain the raw key material") + + opened, err := custodian.DecryptActorKey(did, sealed) + require.NoError(t, err) + assert.True(t, bytes.Equal(key.Bytes(), opened.Bytes()), "decrypted key must equal the original") +} + +func TestCustodian_CiphertextBoundToDID(t *testing.T) { + custodian := testCustodian(t) + key, err := atcrypto.GeneratePrivateKeyK256() + require.NoError(t, err) + + sealed, err := custodian.EncryptActorKey("did:plc:ewvi7nxzyoun6zhxrhs64oiz", key) + require.NoError(t, err) + + _, err = custodian.DecryptActorKey("did:plc:44ybard66vv44zksje25o7dz", sealed) + require.Error(t, err, "a key sealed for one DID must not open under another (AAD binding)") +} + +func TestCustodian_WrongKEKFails(t *testing.T) { + custodian := testCustodian(t) + key, err := atcrypto.GeneratePrivateKeyK256() + require.NoError(t, err) + + const did = "did:plc:ewvi7nxzyoun6zhxrhs64oiz" + sealed, err := custodian.EncryptActorKey(did, key) + require.NoError(t, err) + + otherKEK := sha256.Sum256([]byte("a-different-kek")) + other, err := NewCustodian(otherKEK[:]) + require.NoError(t, err) + _, err = other.DecryptActorKey(did, sealed) + require.Error(t, err) +} + +func TestCustodian_RejectsUnknownVersionAndTruncation(t *testing.T) { + custodian := testCustodian(t) + key, err := atcrypto.GeneratePrivateKeyK256() + require.NoError(t, err) + + const did = "did:plc:ewvi7nxzyoun6zhxrhs64oiz" + sealed, err := custodian.EncryptActorKey(did, key) + require.NoError(t, err) + + tampered := append([]byte{}, sealed...) + tampered[0] = 99 + _, err = custodian.DecryptActorKey(did, tampered) + require.Error(t, err, "unknown ciphertext version must be rejected") + + _, err = custodian.DecryptActorKey(did, sealed[:8]) + require.Error(t, err, "truncated ciphertext must be rejected") +} + +func TestCustodian_CrossContextAADRejected(t *testing.T) { + // The AAD strings bind each ciphertext to its purpose, not just its + // owner: a sealed rotation key pasted into bridged_actors.signing_key + // (or an actor key into the service_keys rotation row) must fail to + // open, even under the same KEK. + custodian := testCustodian(t) + key, err := atcrypto.GeneratePrivateKeyK256() + require.NoError(t, err) + const did = "did:plc:ewvi7nxzyoun6zhxrhs64oiz" + + // Rotation-key ciphertext must not open as an actor key. + rotationSealed, err := custodian.seal(key.Bytes(), []byte(rotationKeyAAD)) + require.NoError(t, err) + _, err = custodian.DecryptActorKey(did, rotationSealed) + require.Error(t, err, + "a sealed rotation key must not open as an actor signing key") + + // Actor-key ciphertext must not open as the rotation key (via the real + // production open path, decryptRotationKey). + actorSealed, err := custodian.EncryptActorKey(did, key) + require.NoError(t, err) + _, err = decryptRotationKey(custodian, actorSealed) + require.Error(t, err, + "a sealed actor key must not open as the rotation key") +} + +// The rotation-key and ActorKeys tests run against real postgres (skipped +// without TIDEPOOL_TEST_DATABASE_URL, per repo convention). + +func TestLoadOrCreateRotationKey_PersistsAcrossLoads(t *testing.T) { + database := testutil.DB(t) + testutil.Truncate(t, database, "service_keys") + keys := store.NewServiceKeys(database) + custodian := testCustodian(t) + ctx := t.Context() + + first, err := LoadOrCreateRotationKey(ctx, keys, custodian) + require.NoError(t, err) + second, err := LoadOrCreateRotationKey(ctx, keys, custodian) + require.NoError(t, err) + + assert.True(t, bytes.Equal(first.Bytes(), second.Bytes()), + "second load must return the persisted key, not a fresh one") + + // The stored bytes must be sealed, not the raw key. + stored, err := keys.Get(ctx, RotationKeyName) + require.NoError(t, err) + assert.False(t, bytes.Contains(stored.PrivateKeyPEM, first.Bytes()), + "rotation key must be encrypted at rest") +} + +func TestActorKeys_SigningKeyLifecycle(t *testing.T) { + database := testutil.DB(t) + testutil.Truncate(t, database, "bridged_actors") + actors := store.NewBridgedActors(database) + custodian := testCustodian(t) + actorKeys := NewActorKeys(actors, custodian) + ctx := t.Context() + + signing, err := atcrypto.GeneratePrivateKeyK256() + require.NoError(t, err) + const did = "did:plc:ewvi7nxzyoun6zhxrhs64oiz" + sealed, err := custodian.EncryptActorKey(did, signing) + require.NoError(t, err) + + const apActorID = "https://lemmy.world/u/alice" + _, err = actors.UpsertActor(ctx, store.BridgedActor{ + APActorID: apActorID, + ActorType: store.ActorTypePerson, + DID: did, + Handle: "alice.lemmy-world.tidepool.example", + SigningKeyEncrypted: sealed, + ConsentState: store.ConsentStateOK, + }) + require.NoError(t, err) + + got, err := actorKeys.SigningKey(ctx, did, repo.KeyUseWrite) + require.NoError(t, err) + gotK256, ok := got.(*atcrypto.PrivateKeyK256) + require.True(t, ok) + assert.True(t, bytes.Equal(signing.Bytes(), gotK256.Bytes())) + + // Unknown DID → not found. + _, err = actorKeys.SigningKey(ctx, "did:plc:44ybard66vv44zksje25o7dz", repo.KeyUseWrite) + assert.True(t, errors.IsNotFound(err)) + + // Tombstoned actor → frozen for writes: no key for KeyUseWrite. + require.NoError(t, actors.SetConsentState(ctx, apActorID, store.ConsentStateDeleted)) + _, err = actorKeys.SigningKey(ctx, did, repo.KeyUseWrite) + assert.True(t, errors.IsTombstoned(err), + "tombstoned actors' repos are frozen via key custody") + assert.False(t, errors.IsNotFound(err), "tombstoned must not read as missing") + + // ... but deletes still get the key: scrubbing a deleted actor's + // records must always be possible (task 05's Delete(Actor) flow). + gotDel, err := actorKeys.SigningKey(ctx, did, repo.KeyUseDelete) + require.NoError(t, err, + "tombstoned actor's key must still be released for KeyUseDelete") + gotDelK256, ok := gotDel.(*atcrypto.PrivateKeyK256) + require.True(t, ok) + assert.True(t, bytes.Equal(signing.Bytes(), gotDelK256.Bytes())) +} diff --git a/internal/identity/minter.go b/internal/identity/minter.go new file mode 100644 index 0000000..f4a104a --- /dev/null +++ b/internal/identity/minter.go @@ -0,0 +1,398 @@ +package identity + +import ( + "bytes" + "context" + "crypto/sha256" + "encoding/base32" + "encoding/base64" + "encoding/hex" + "encoding/json" + "fmt" + "io" + "log/slog" + "net/http" + "net/url" + "strings" + + "github.com/bluesky-social/indigo/atproto/atcrypto" + "github.com/bluesky-social/indigo/atproto/atdata" + "github.com/bluesky-social/indigo/atproto/syntax" + + "tidepool/internal/errors" + "tidepool/internal/store" +) + +// maxHandleAttempts bounds collision suffixing: base handle plus -2..-N. +const maxHandleAttempts = 50 + +// maxPLCErrorBody caps how much of a PLC directory error response is read +// back for the error message. +const maxPLCErrorBody = 4 << 10 + +// MintRequest describes the fediverse actor an identity is minted for. +type MintRequest struct { + // ActorType selects person or group; the handle scheme is the same for + // both (name.instance-with-dashes.bridge-hostname), the type only + // drives logging. + ActorType store.ActorType + // PreferredUsername is the AP preferredUsername (e.g. "alice", + // "technology"). + PreferredUsername string + // Instance is the actor's home host (e.g. "lemmy.world"). + Instance string +} + +// Identity is a freshly minted did:plc plus its escrowed key material. The +// caller (tasks 05/06) persists it via store.BridgedActors.UpsertActor — +// SigningKeyEncrypted goes into the signing_key column as-is. +type Identity struct { + DID string + Handle string + // DIDKey is the did:key encoding of the actor's verification + // (signing) public key, as registered in the PLC operation. + DIDKey string + // SigningKeyEncrypted is the actor's secp256k1 private key sealed with + // the bridge KEK, bound to DID. + SigningKeyEncrypted []byte +} + +// Minter creates did:plc identities on a PLC directory for bridged actors. +// +// Per PLAN.md: each actor gets its own secp256k1 signing key (the +// verification key in the DID document); the bridge's single escrow +// rotation key is the only rotation key, which is what makes later +// claiming/migration possible (the bridge can sign a PLC op handing the +// identity over). The PDS endpoint in every DID doc is the bridge itself. +type Minter struct { + plcURL string + bridgeHostname string + rotationKey *atcrypto.PrivateKeyK256 + custodian *Custodian + actors store.BridgedActors + httpClient *http.Client + userAgent string + logger *slog.Logger +} + +// MinterOptions configures NewMinter. All fields except HTTPClient are +// required. +type MinterOptions struct { + // PLCDirectoryURL is the directory ops are POSTed to + // (config.PLCDirectoryURL, e.g. https://plc.directory). + PLCDirectoryURL string + // BridgeHostname anchors the handle space and the PDS endpoint + // (config.BridgeHostname). + BridgeHostname string + // RotationKey is the bridge escrow rotation key + // (LoadOrCreateRotationKey). + RotationKey *atcrypto.PrivateKeyK256 + // Custodian seals the per-actor signing keys. + Custodian *Custodian + // Actors is consulted for handle-collision suffixing. + Actors store.BridgedActors + // HTTPClient makes the directory requests. Production wires + // ap.NewGuardedHTTPClient(cfg.AllowPrivateAddresses, 0) so PLC egress + // shares the AP client's SSRF guard; tests hitting 127.0.0.1 pass a + // guard-disabled client the same way. + HTTPClient *http.Client + // UserAgent is sent on every directory request (config.UserAgent — + // same identification convention as the AP client). Defaults to the AP + // client's fallback when empty. + UserAgent string + Logger *slog.Logger +} + +// NewMinter validates the options and builds a Minter. +func NewMinter(opts MinterOptions) (*Minter, error) { + u, err := url.Parse(opts.PLCDirectoryURL) + if err != nil || (u.Scheme != "http" && u.Scheme != "https") || u.Host == "" { + return nil, errors.NewValidationError("plc_directory_url", + fmt.Sprintf("%q is not an absolute http(s) URL", opts.PLCDirectoryURL)) + } + if opts.BridgeHostname == "" { + return nil, errors.NewValidationError("bridge_hostname", "must not be empty") + } + if opts.RotationKey == nil { + return nil, errors.NewValidationError("rotation_key", "must not be nil") + } + if opts.Custodian == nil { + return nil, errors.NewValidationError("custodian", "must not be nil") + } + if opts.Actors == nil { + return nil, errors.NewValidationError("actors", "must not be nil") + } + if opts.HTTPClient == nil { + return nil, errors.NewValidationError("http_client", + "must be provided (use ap.NewGuardedHTTPClient so PLC egress is SSRF-guarded)") + } + logger := opts.Logger + if logger == nil { + logger = slog.Default() + } + userAgent := opts.UserAgent + if userAgent == "" { + userAgent = "tidepool/0.1" // matches internal/ap's client fallback + } + return &Minter{ + plcURL: strings.TrimRight(opts.PLCDirectoryURL, "/"), + bridgeHostname: strings.ToLower(opts.BridgeHostname), + rotationKey: opts.RotationKey, + custodian: opts.Custodian, + actors: opts.Actors, + httpClient: opts.HTTPClient, + userAgent: userAgent, + logger: logger, + }, nil +} + +// MintActor generates a signing keypair, picks a free bridged handle, signs +// the genesis PLC operation with the escrow rotation key, registers it with +// the directory, and returns the identity with the sealed signing key. +// +// It does NOT write bridged_actors: callers upsert the returned Identity +// themselves (they own consent state). A concurrent mint racing the same +// handle is caught by the bridged_actors handle unique index at that point. +// +// Failure semantics: the DID is derived locally and the signing key is +// sealed BEFORE the operation is submitted, so a key-custody failure can +// never orphan a registered DID. A failure during submission, however, can +// still leave a live DID on the directory (registration is irreversible and +// e.g. a timeout may fire after the directory processed the op) with no +// bridged_actors row. Such orphans are logged at error level with did and +// handle so they can be found; making a retry after the resulting handle +// collision reuse the registered DID is deferred to tasks 05/06. +func (m *Minter) MintActor(ctx context.Context, req MintRequest) (*Identity, error) { + if !req.ActorType.Valid() { + return nil, errors.NewValidationError("actor_type", fmt.Sprintf("unknown actor type %q", req.ActorType)) + } + handle, err := m.availableHandle(ctx, req.PreferredUsername, req.Instance) + if err != nil { + return nil, err + } + + signingKey, err := atcrypto.GeneratePrivateKeyK256() + if err != nil { + return nil, fmt.Errorf("identity: generate signing key: %w", err) + } + signingPub, err := signingKey.PublicKey() + if err != nil { + return nil, fmt.Errorf("identity: derive signing public key: %w", err) + } + rotationPub, err := m.rotationKey.PublicKey() + if err != nil { + return nil, fmt.Errorf("identity: derive rotation public key: %w", err) + } + + op, err := m.signGenesisOp(genesisOperation( + rotationPub.DIDKey(), signingPub.DIDKey(), handle, "https://"+m.bridgeHostname)) + if err != nil { + return nil, err + } + did, err := didForOperation(op) + if err != nil { + return nil, err + } + + // Seal the key BEFORE submitting the irreversible PLC op: a custody + // failure here aborts the mint with nothing registered anywhere. + sealed, err := m.custodian.EncryptActorKey(did, signingKey) + if err != nil { + return nil, err + } + + if err := m.submitOperation(ctx, did, op); err != nil { + // The directory may have processed the op even though the call + // failed (e.g. timeout after registration): treat this as a + // potential orphaned DID and make it findable in the logs. + m.logger.Error("PLC mint failed at submission; DID may be orphaned on the directory", + "did", did, "handle", handle, "error", err) + return nil, fmt.Errorf("identity: mint %s (handle %q): %w", did, handle, err) + } + + m.logger.Info("minted did:plc for bridged actor", + "did", did, "handle", handle, "actor_type", string(req.ActorType), "instance", req.Instance) + + return &Identity{ + DID: did, + Handle: handle, + DIDKey: signingPub.DIDKey(), + SigningKeyEncrypted: sealed, + }, nil +} + +// availableHandle builds the bridged handle and suffixes it (-2, -3, ...) +// until it does not collide with an already-assigned handle. +func (m *Minter) availableHandle(ctx context.Context, username, instance string) (string, error) { + base, instanceLabel, err := handleLabels(username, instance) + if err != nil { + return "", err + } + for attempt := 1; attempt <= maxHandleAttempts; attempt++ { + name := base + if attempt > 1 { + name = suffixLabel(base, fmt.Sprintf("-%d", attempt)) + } + candidate := fmt.Sprintf("%s.%s.%s", name, instanceLabel, m.bridgeHostname) + if _, err := syntax.ParseHandle(candidate); err != nil { + return "", errors.NewValidationError("handle", + fmt.Sprintf("derived handle %q is not a valid atproto handle: %v", candidate, err)) + } + _, err := m.actors.GetByHandle(ctx, candidate) + if errors.IsNotFound(err) { + return candidate, nil + } + if err != nil { + return "", fmt.Errorf("identity: check handle availability for %q: %w", candidate, err) + } + // Taken; try the next suffix. + } + return "", errors.NewValidationError("handle", + fmt.Sprintf("no free handle for %s@%s after %d attempts", username, instance, maxHandleAttempts)) +} + +// handleLabels normalizes the AP username and instance host into DNS labels +// per the locked handle scheme: dots separate what @/! separated on the +// fediverse side, and the instance's own dots become dashes +// (alice@lemmy.world → alice.lemmy-world.). Characters that cannot +// appear in a handle label (Lemmy allows underscores in usernames) map to +// dashes too. +func handleLabels(username, instance string) (name string, instanceLabel string, err error) { + name = normalizeLabel(username) + if name == "" && username != "" { + // All-CJK/emoji usernames normalize to nothing; fall back to a + // deterministic hash-derived label so those actors still bridge. + name = fallbackLabel(username) + } + if name == "" { + return "", "", errors.NewValidationError("preferred_username", + fmt.Sprintf("%q normalizes to an empty handle label", username)) + } + instanceLabel = normalizeLabel(strings.ReplaceAll(instance, ".", "-")) + if instanceLabel == "" { + return "", "", errors.NewValidationError("instance", + fmt.Sprintf("%q normalizes to an empty handle label", instance)) + } + return name, instanceLabel, nil +} + +// fallbackLabel derives a stable DNS label for a string with no characters +// representable in [a-z0-9]: "u" + the first 10 hex chars of its sha256. +func fallbackLabel(s string) string { + sum := sha256.Sum256([]byte(s)) + return "u" + hex.EncodeToString(sum[:])[:10] +} + +// suffixLabel appends suffix to base, truncating base first so the result +// still fits the 63-char DNS label limit (normalizeLabel truncates to 63 +// BEFORE suffixing, so "-2".."-50" would otherwise overflow the label). +func suffixLabel(base, suffix string) string { + if max := 63 - len(suffix); len(base) > max { + base = strings.TrimRight(base[:max], "-") + } + return base + suffix +} + +// normalizeLabel lowercases and maps a string into the DNS-label alphabet +// [a-z0-9-], collapsing runs of unrepresentable characters into single +// dashes and trimming dashes from the ends. Truncated to 63 chars (DNS +// label limit; atproto handles inherit it). +func normalizeLabel(s string) string { + var b strings.Builder + lastDash := true // suppress leading dash + for _, r := range strings.ToLower(s) { + switch { + case r >= 'a' && r <= 'z' || r >= '0' && r <= '9': + b.WriteRune(r) + lastDash = false + default: + if !lastDash { + b.WriteByte('-') + lastDash = true + } + } + } + label := strings.TrimRight(b.String(), "-") + if len(label) > 63 { + label = strings.TrimRight(label[:63], "-") + } + return label +} + +// genesisOperation builds the (unsigned) did:plc genesis operation. This is +// a PLC *operation*, not a DID document — same shape arroba/bridgy-fed +// publish: one escrow rotation key, the actor's own verification key, the +// bridged handle in alsoKnownAs, and the bridge as the PDS endpoint. +// https://github.com/did-method-plc/did-method-plc#operation-serialization-signing-and-validation +func genesisOperation(rotationDIDKey, signingDIDKey, handle, pdsEndpoint string) map[string]any { + return map[string]any{ + "type": "plc_operation", + "rotationKeys": []any{rotationDIDKey}, + "verificationMethods": map[string]any{ + "atproto": signingDIDKey, + }, + "alsoKnownAs": []any{"at://" + handle}, + "services": map[string]any{ + "atproto_pds": map[string]any{ + "type": "AtprotoPersonalDataServer", + "endpoint": pdsEndpoint, + }, + }, + "prev": nil, + } +} + +// signGenesisOp signs the operation with the escrow rotation key: the +// signature is over the DAG-CBOR encoding of the op without its sig field, +// base64url (no padding) encoded, per the PLC spec. +func (m *Minter) signGenesisOp(op map[string]any) (map[string]any, error) { + unsigned, err := atdata.MarshalCBOR(op) + if err != nil { + return nil, fmt.Errorf("identity: encode PLC op: %w", err) + } + sig, err := m.rotationKey.HashAndSign(unsigned) + if err != nil { + return nil, fmt.Errorf("identity: sign PLC op: %w", err) + } + op["sig"] = base64.RawURLEncoding.EncodeToString(sig) + return op, nil +} + +// didForOperation derives the did:plc from the signed genesis operation: +// base32(sha256(dag-cbor(signed op))) truncated to 24 chars, lowercased. +func didForOperation(signedOp map[string]any) (string, error) { + encoded, err := atdata.MarshalCBOR(signedOp) + if err != nil { + return "", fmt.Errorf("identity: encode signed PLC op: %w", err) + } + sum := sha256.Sum256(encoded) + hash := strings.ToLower(base32.StdEncoding.EncodeToString(sum[:])) + return "did:plc:" + hash[:24], nil +} + +// submitOperation POSTs the signed operation to the PLC directory. +func (m *Minter) submitOperation(ctx context.Context, did string, signedOp map[string]any) error { + body, err := json.Marshal(signedOp) + if err != nil { + return fmt.Errorf("identity: marshal PLC op: %w", err) + } + req, err := http.NewRequestWithContext(ctx, http.MethodPost, + m.plcURL+"/"+url.PathEscape(did), bytes.NewReader(body)) + if err != nil { + return fmt.Errorf("identity: build PLC request: %w", err) + } + req.Header.Set("Content-Type", "application/json") + req.Header.Set("User-Agent", m.userAgent) + + resp, err := m.httpClient.Do(req) + if err != nil { + return fmt.Errorf("identity: POST PLC op for %s: %w", did, err) + } + defer func() { _ = resp.Body.Close() }() + if resp.StatusCode < 200 || resp.StatusCode > 299 { + msg, _ := io.ReadAll(io.LimitReader(resp.Body, maxPLCErrorBody)) + return fmt.Errorf("identity: PLC directory rejected op for %s: status %d: %s", + did, resp.StatusCode, strings.TrimSpace(string(msg))) + } + return nil +} diff --git a/internal/identity/minter_test.go b/internal/identity/minter_test.go new file mode 100644 index 0000000..d2dbb6e --- /dev/null +++ b/internal/identity/minter_test.go @@ -0,0 +1,348 @@ +package identity + +import ( + "context" + "encoding/json" + "fmt" + "io" + "net" + "net/http" + "net/url" + "os" + "regexp" + "strings" + "testing" + "time" + + "github.com/bluesky-social/indigo/atproto/atcrypto" + "github.com/bluesky-social/indigo/atproto/atdata" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "tidepool/internal/ap" + "tidepool/internal/errors" + "tidepool/internal/store" + "tidepool/internal/testutil" +) + +func TestHandleLabels(t *testing.T) { + tests := []struct { + name string + username string + instance string + wantName string + wantInstance string + wantErr bool + }{ + {name: "community", username: "technology", instance: "lemmy.world", wantName: "technology", wantInstance: "lemmy-world"}, + {name: "user", username: "alice", instance: "lemmy.world", wantName: "alice", wantInstance: "lemmy-world"}, + {name: "underscores become dashes", username: "cool_user", instance: "lemmy.world", wantName: "cool-user", wantInstance: "lemmy-world"}, + {name: "uppercase lowered", username: "Alice", instance: "Lemmy.World", wantName: "alice", wantInstance: "lemmy-world"}, + {name: "multi-label instance", username: "news", instance: "lemmy.sdf.org", wantName: "news", wantInstance: "lemmy-sdf-org"}, + {name: "weird chars collapse", username: "a__b!!c", instance: "lemmy.world", wantName: "a-b-c", wantInstance: "lemmy-world"}, + {name: "trailing junk trimmed", username: "alice_", instance: "lemmy.world", wantName: "alice", wantInstance: "lemmy-world"}, + // Usernames with no representable characters fall back to a + // deterministic hash label instead of failing the mint. + {name: "unrepresentable username", username: "___", instance: "lemmy.world", wantName: fallbackLabel("___"), wantInstance: "lemmy-world"}, + {name: "cjk username", username: "日本語ユーザー", instance: "lemmy.world", wantName: fallbackLabel("日本語ユーザー"), wantInstance: "lemmy-world"}, + {name: "emoji username", username: "🦀🦀", instance: "lemmy.world", wantName: fallbackLabel("🦀🦀"), wantInstance: "lemmy-world"}, + {name: "empty username", username: "", instance: "lemmy.world", wantErr: true}, + {name: "empty instance", username: "alice", instance: "", wantErr: true}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + name, instance, err := handleLabels(tt.username, tt.instance) + if tt.wantErr { + require.Error(t, err) + assert.True(t, errors.IsValidation(err)) + return + } + require.NoError(t, err) + assert.Equal(t, tt.wantName, name) + assert.Equal(t, tt.wantInstance, instance) + }) + } +} + +func TestNormalizeLabel_TruncatesTo63(t *testing.T) { + long := normalizeLabel(bytesRepeat('a', 80)) + assert.Len(t, long, 63) +} + +func TestFallbackLabel_DeterministicAndValid(t *testing.T) { + a := fallbackLabel("日本語ユーザー") + assert.Equal(t, a, fallbackLabel("日本語ユーザー"), "fallback must be deterministic") + assert.NotEqual(t, a, fallbackLabel("другой"), "different inputs must differ") + assert.Regexp(t, `^u[0-9a-f]{10}$`, a) +} + +func TestSuffixLabel_KeepsDNSLabelLimit(t *testing.T) { + // normalizeLabel truncates to 63 BEFORE suffixing; the suffix must eat + // into the base, not overflow the label. + base := normalizeLabel(bytesRepeat('a', 80)) + require.Len(t, base, 63) + + for _, suffix := range []string{"-2", "-10", "-50"} { + got := suffixLabel(base, suffix) + assert.LessOrEqual(t, len(got), 63, "suffix %s must not overflow the label", suffix) + assert.True(t, strings.HasSuffix(got, suffix)) + } + + // Short bases are left alone. + assert.Equal(t, "alice-2", suffixLabel("alice", "-2")) +} + +func bytesRepeat(b byte, n int) string { + out := make([]byte, n) + for i := range out { + out[i] = b + } + return string(out) +} + +// TestGenesisOpEncoding pins the DAG-CBOR assumptions the PLC op signing +// depends on: nil encodes as CBOR null (the genesis op's prev), and the +// derived DID is deterministic with the did:plc shape. +func TestGenesisOpEncoding(t *testing.T) { + encoded, err := atdata.MarshalCBOR(map[string]any{"prev": nil}) + require.NoError(t, err) + // A1 (map of 1) 64 "prev" F6 (null) + assert.Equal(t, []byte{0xa1, 0x64, 'p', 'r', 'e', 'v', 0xf6}, encoded, + "nil map values must encode as DAG-CBOR null") + + op := genesisOperation("did:key:zRotation", "did:key:zSigning", + "technology.lemmy-world.tidepool.example", "https://tidepool.example") + op["sig"] = "fakesig" + did1, err := didForOperation(op) + require.NoError(t, err) + did2, err := didForOperation(op) + require.NoError(t, err) + assert.Equal(t, did1, did2, "DID derivation must be deterministic") + assert.Regexp(t, regexp.MustCompile(`^did:plc:[a-z2-7]{24}$`), did1) +} + +// --- PLC directory integration --- +// +// These tests exercise a REAL local PLC directory (did-method-plc). They +// never touch the public https://plc.directory: testPLCURL hard-fails on +// any non-loopback host. They skip under -short and when the local +// directory is unreachable (docker compose --profile plc up -d, or the +// Coves dev PLC on the same port). + +const defaultTestPLCURL = "http://localhost:3002" + +// testPLCURL returns the PLC directory URL for integration tests, skipping +// when unavailable and refusing to run against anything but loopback. +func testPLCURL(t *testing.T) string { + t.Helper() + if testing.Short() { + t.Skip("short mode: skipping PLC directory integration test") + } + plcURL := os.Getenv("TIDEPOOL_TEST_PLC_URL") + if plcURL == "" { + plcURL = defaultTestPLCURL + } + + // Safety rail: minting tests create DIDs; they must never spam the + // public directory. Fail (not skip) so a misconfigured environment is + // loud. + parsed, err := url.Parse(plcURL) + require.NoError(t, err, "TIDEPOOL_TEST_PLC_URL must be a URL") + host := parsed.Hostname() + if ip := net.ParseIP(host); ip != nil { + if !ip.IsLoopback() { + t.Fatalf("refusing to run PLC minting tests against non-loopback %q: "+ + "tests must never create DIDs on a public directory", plcURL) + } + } else if host != "localhost" { + t.Fatalf("refusing to run PLC minting tests against non-localhost %q: "+ + "tests must never create DIDs on a public directory", plcURL) + } + + ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) + defer cancel() + req, err := http.NewRequestWithContext(ctx, http.MethodGet, plcURL+"/_health", nil) + require.NoError(t, err) + resp, err := http.DefaultClient.Do(req) + if err != nil { + t.Skipf("local PLC directory not reachable at %s (start it with "+ + "`docker compose -f docker-compose.dev.yml --profile plc up -d`): %v", plcURL, err) + } + defer func() { _ = resp.Body.Close() }() + if resp.StatusCode != http.StatusOK { + t.Skipf("local PLC directory unhealthy at %s: status %d", plcURL, resp.StatusCode) + } + return plcURL +} + +// testMinter builds a Minter against the local PLC directory and real +// postgres. Handles get a unique per-run bridge hostname so reruns against +// a shared PLC container never collide. +func testMinter(t *testing.T, actors store.BridgedActors) (*Minter, *Custodian, *atcrypto.PrivateKeyK256, string) { + t.Helper() + plcURL := testPLCURL(t) + custodian := testCustodian(t) + rotationKey, err := atcrypto.GeneratePrivateKeyK256() + require.NoError(t, err) + bridgeHostname := fmt.Sprintf("t%d.tidepool.example", time.Now().UnixNano()) + + minter, err := NewMinter(MinterOptions{ + PLCDirectoryURL: plcURL, + BridgeHostname: bridgeHostname, + RotationKey: rotationKey, + Custodian: custodian, + Actors: actors, + // The PLC client shares the AP client's SSRF guard; localhost needs + // the dev/test override, exactly like the httptest-based AP tests. + HTTPClient: ap.NewGuardedHTTPClient(true, 10*time.Second), + }) + require.NoError(t, err) + return minter, custodian, rotationKey, bridgeHostname +} + +func TestMintActor_AgainstRealPLC(t *testing.T) { + database := testutil.DB(t) + testutil.Truncate(t, database, "bridged_actors") + actors := store.NewBridgedActors(database) + minter, custodian, rotationKey, bridgeHostname := testMinter(t, actors) + ctx := t.Context() + + identity, err := minter.MintActor(ctx, MintRequest{ + ActorType: store.ActorTypeGroup, + PreferredUsername: "technology", + Instance: "lemmy.world", + }) + require.NoError(t, err) + + assert.Regexp(t, `^did:plc:[a-z2-7]{24}$`, identity.DID) + assert.Equal(t, "technology.lemmy-world."+bridgeHostname, identity.Handle) + + // The sealed signing key must decrypt back to the key registered as the + // verification key. + signingKey, err := custodian.DecryptActorKey(identity.DID, identity.SigningKeyEncrypted) + require.NoError(t, err) + signingPub, err := signingKey.PublicKey() + require.NoError(t, err) + assert.Equal(t, identity.DIDKey, signingPub.DIDKey()) + + // Resolve the DID document from the directory and verify every claim. + doc := fetchDIDDoc(t, minter.plcURL, identity.DID) + assert.Equal(t, identity.DID, doc.ID) + require.NotEmpty(t, doc.AlsoKnownAs) + assert.Equal(t, "at://"+identity.Handle, doc.AlsoKnownAs[0]) + + require.Len(t, doc.VerificationMethod, 1) + assert.Equal(t, "Multikey", doc.VerificationMethod[0].Type) + assert.Equal(t, identity.DIDKey, "did:key:"+doc.VerificationMethod[0].PublicKeyMultibase, + "the DID doc's verification key must be the per-actor signing key") + + require.Len(t, doc.Service, 1) + assert.Equal(t, "AtprotoPersonalDataServer", doc.Service[0].Type) + assert.Equal(t, "https://"+bridgeHostname, doc.Service[0].ServiceEndpoint, + "the PDS endpoint in the DID doc must be the bridge") + + // The directory's audit log must show the escrow rotation key. + rotationPub, err := rotationKey.PublicKey() + require.NoError(t, err) + log := fetchAuditLog(t, minter.plcURL, identity.DID) + require.NotEmpty(t, log) + assert.Contains(t, log[0].Operation.RotationKeys, rotationPub.DIDKey(), + "genesis op must carry the bridge escrow rotation key") +} + +func TestMintActor_HandleCollisionSuffixes(t *testing.T) { + database := testutil.DB(t) + testutil.Truncate(t, database, "bridged_actors") + actors := store.NewBridgedActors(database) + minter, _, _, bridgeHostname := testMinter(t, actors) + ctx := t.Context() + + req := MintRequest{ + ActorType: store.ActorTypePerson, + PreferredUsername: "alice", + Instance: "lemmy.world", + } + + first, err := minter.MintActor(ctx, req) + require.NoError(t, err) + assert.Equal(t, "alice.lemmy-world."+bridgeHostname, first.Handle) + + // Persist the first identity so its handle is taken (the minter's + // availability check reads bridged_actors). + _, err = actors.UpsertActor(ctx, store.BridgedActor{ + APActorID: "https://lemmy.world/u/alice", + ActorType: store.ActorTypePerson, + DID: first.DID, + Handle: first.Handle, + SigningKeyEncrypted: first.SigningKeyEncrypted, + ConsentState: store.ConsentStateOK, + }) + require.NoError(t, err) + + // A different AP actor normalizing to the same handle (e.g. + // alice@lemmy.world vs Alice@lemmy.world after a rename) gets -2. + second, err := minter.MintActor(ctx, req) + require.NoError(t, err) + assert.Equal(t, "alice-2.lemmy-world."+bridgeHostname, second.Handle) + assert.NotEqual(t, first.DID, second.DID) +} + +// didDoc is the subset of a PLC DID document the tests verify. +type didDoc struct { + ID string `json:"id"` + AlsoKnownAs []string `json:"alsoKnownAs"` + VerificationMethod []struct { + ID string `json:"id"` + Type string `json:"type"` + PublicKeyMultibase string `json:"publicKeyMultibase"` + } `json:"verificationMethod"` + Service []struct { + ID string `json:"id"` + Type string `json:"type"` + ServiceEndpoint string `json:"serviceEndpoint"` + } `json:"service"` +} + +func fetchDIDDoc(t *testing.T, plcURL, did string) didDoc { + t.Helper() + var doc didDoc + fetchJSON(t, plcURL+"/"+did, &doc) + return doc +} + +type auditEntry struct { + Operation struct { + RotationKeys []string `json:"rotationKeys"` + } `json:"operation"` +} + +func fetchAuditLog(t *testing.T, plcURL, did string) []auditEntry { + t.Helper() + var log []auditEntry + fetchJSON(t, plcURL+"/"+did+"/log/audit", &log) + return log +} + +func fetchJSON(t *testing.T, rawURL string, out any) { + t.Helper() + ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) + defer cancel() + req, err := http.NewRequestWithContext(ctx, http.MethodGet, rawURL, nil) + require.NoError(t, err) + resp, err := http.DefaultClient.Do(req) + require.NoError(t, err) + defer func() { _ = resp.Body.Close() }() + body, err := io.ReadAll(resp.Body) + require.NoError(t, err) + require.Equal(t, http.StatusOK, resp.StatusCode, "GET %s: %s", rawURL, body) + require.NoError(t, json.Unmarshal(body, out)) +} + +// Guard against accidental reintroduction of a live-directory default: the +// minter must always be told its directory explicitly. +func TestNewMinter_RequiresExplicitConfig(t *testing.T) { + _, err := NewMinter(MinterOptions{}) + require.Error(t, err) + assert.True(t, errors.IsValidation(err)) + assert.NotContains(t, err.Error(), "plc.directory") +} diff --git a/internal/repo/blocks.go b/internal/repo/blocks.go new file mode 100644 index 0000000..3fae48e --- /dev/null +++ b/internal/repo/blocks.go @@ -0,0 +1,142 @@ +package repo + +import ( + "context" + "database/sql" + stderrors "errors" + "fmt" + + blockformat "github.com/ipfs/go-block-format" + "github.com/ipfs/go-cid" + ipld "github.com/ipfs/go-ipld-format" + "github.com/multiformats/go-multihash" +) + +// dagCBORPrefix is the CID shape every repo block uses: CIDv1, dag-cbor, +// sha2-256. +var dagCBORPrefix = cid.Prefix{ + Version: 1, + Codec: cid.DagCBOR, + MhType: multihash.SHA2_256, + MhLength: 32, +} + +// cidForBlock computes the dag-cbor CID for raw block bytes. +func cidForBlock(data []byte) (cid.Cid, error) { + c, err := dagCBORPrefix.Sum(data) + if err != nil { + return cid.Undef, fmt.Errorf("repo: compute cid: %w", err) + } + return c, nil +} + +// txBlockSource reads a single DID's blocks from postgres inside the commit +// transaction. It implements mst.MSTBlockSource / repo.RepoBlockSource +// (Get only). Misses return ipld.ErrNotFound so indigo's tree loader treats +// them per its own semantics — for our own repos every block is expected to +// be present, and a partial tree will surface as mst.ErrPartialTree on the +// next mutation. +type txBlockSource struct { + tx *sql.Tx + did string +} + +func (s *txBlockSource) Get(ctx context.Context, c cid.Cid) (blockformat.Block, error) { + var raw []byte + err := s.tx.QueryRowContext(ctx, + `SELECT bytes FROM blocks WHERE did = $1 AND cid = $2`, + s.did, c.String()).Scan(&raw) + if stderrors.Is(err, sql.ErrNoRows) { + return nil, ipld.ErrNotFound{Cid: c} + } + if err != nil { + return nil, fmt.Errorf("repo: read block %s for %s: %w", c, s.did, err) + } + return blockformat.NewBlockWithCid(raw, c) +} + +// memBlockstore is an ordered, in-memory blockstore capturing the blocks a +// commit produces (MST diff nodes via WriteDiffBlocks, plus record and +// commit blocks added directly). It implements the full +// go-ipfs-blockstore.Blockstore interface because indigo's WriteDiffBlocks +// asks for it, but only Put/Get/Has see real use. +type memBlockstore struct { + order []cid.Cid + blocks map[cid.Cid]blockformat.Block +} + +func newMemBlockstore() *memBlockstore { + return &memBlockstore{blocks: make(map[cid.Cid]blockformat.Block)} +} + +// ordered returns the captured blocks in insertion order. +func (m *memBlockstore) ordered() []blockformat.Block { + out := make([]blockformat.Block, 0, len(m.order)) + for _, c := range m.order { + out = append(out, m.blocks[c]) + } + return out +} + +func (m *memBlockstore) Put(_ context.Context, blk blockformat.Block) error { + c := blk.Cid() + if _, ok := m.blocks[c]; !ok { + m.order = append(m.order, c) + m.blocks[c] = blk + } + return nil +} + +func (m *memBlockstore) PutMany(ctx context.Context, blks []blockformat.Block) error { + for _, blk := range blks { + if err := m.Put(ctx, blk); err != nil { + return err + } + } + return nil +} + +func (m *memBlockstore) Get(_ context.Context, c cid.Cid) (blockformat.Block, error) { + blk, ok := m.blocks[c] + if !ok { + return nil, ipld.ErrNotFound{Cid: c} + } + return blk, nil +} + +func (m *memBlockstore) Has(_ context.Context, c cid.Cid) (bool, error) { + _, ok := m.blocks[c] + return ok, nil +} + +func (m *memBlockstore) GetSize(_ context.Context, c cid.Cid) (int, error) { + blk, ok := m.blocks[c] + if !ok { + return 0, ipld.ErrNotFound{Cid: c} + } + return len(blk.RawData()), nil +} + +func (m *memBlockstore) DeleteBlock(_ context.Context, c cid.Cid) error { + if _, ok := m.blocks[c]; ok { + delete(m.blocks, c) + for i, oc := range m.order { + if oc.Equals(c) { + m.order = append(m.order[:i], m.order[i+1:]...) + break + } + } + } + return nil +} + +func (m *memBlockstore) AllKeysChan(_ context.Context) (<-chan cid.Cid, error) { + ch := make(chan cid.Cid, len(m.order)) + for _, c := range m.order { + ch <- c + } + close(ch) + return ch, nil +} + +func (m *memBlockstore) HashOnRead(bool) {} diff --git a/internal/repo/events.go b/internal/repo/events.go new file mode 100644 index 0000000..a64d23c --- /dev/null +++ b/internal/repo/events.go @@ -0,0 +1,198 @@ +package repo + +import ( + "bytes" + "context" + "database/sql" + "encoding/json" + "fmt" + + indigorepo "github.com/bluesky-social/indigo/atproto/repo" + + blockformat "github.com/ipfs/go-block-format" + "github.com/ipfs/go-cid" + car "github.com/ipld/go-car" + carutil "github.com/ipld/go-car/util" +) + +// OpAction is the kind of record mutation an Op describes. The values +// match the com.atproto.sync.subscribeRepos#repoOp action enum. +type OpAction string + +const ( + OpActionCreate OpAction = "create" + OpActionUpdate OpAction = "update" + OpActionDelete OpAction = "delete" +) + +// Op is one record mutation inside a commit, stored as JSON in +// firehose_events.ops. Task 04 maps these onto +// com.atproto.sync.subscribeRepos#repoOp (action/path/cid, with prev +// carrying the sync-v1.1 previous record CID for updates and deletes). +type Op struct { + Action OpAction `json:"action"` + Path string `json:"path"` // {collection}/{rkey} + CID string `json:"cid,omitempty"` + Prev string `json:"prev,omitempty"` +} + +// opFromIndigoOp converts indigo's Operation to the stored form. +func opFromIndigoOp(op *indigorepo.Operation) Op { + out := Op{Path: op.Path} + switch { + case op.IsCreate(): + out.Action = OpActionCreate + case op.IsUpdate(): + out.Action = OpActionUpdate + case op.IsDelete(): + out.Action = OpActionDelete + } + if op.Value != nil { + out.CID = op.Value.String() + } + if op.Prev != nil { + out.Prev = op.Prev.String() + } + return out +} + +// firehoseEvent is one row of the durable subscribeRepos backlog. +type firehoseEvent struct { + did string + commitCID cid.Cid + prevData *cid.Cid // MST root before this commit; nil on genesis + sinceRev string // previous commit's rev; empty on genesis → NULL + rev string + ops []Op + // blocks is the CAR slice content: the blocks new in this commit + // (MST diff nodes, record blocks, and the commit block itself). + blocks []blockformat.Block +} + +// appendFirehoseEvent inserts the event row inside the commit transaction — +// commit and event are atomic by construction, so the stream can never miss +// a commit or observe one that later rolled back. It returns the seq cursor +// postgres assigned to the event. +func appendFirehoseEvent(ctx context.Context, tx *sql.Tx, ev firehoseEvent) (int64, error) { + carSlice, err := writeCARSlice(ev.commitCID, ev.blocks) + if err != nil { + return 0, err + } + opsJSON, err := json.Marshal(ev.ops) + if err != nil { + return 0, fmt.Errorf("repo: marshal ops: %w", err) + } + var prevData *string + if ev.prevData != nil { + s := ev.prevData.String() + prevData = &s + } + var sinceRev *string + if ev.sinceRev != "" { + sinceRev = &ev.sinceRev + } + var seq int64 + if err := tx.QueryRowContext(ctx, ` + INSERT INTO firehose_events (did, commit_cid, prev_data_cid, since_rev, rev, ops, car) + VALUES ($1, $2, $3, $4, $5, $6, $7) + RETURNING seq`, + ev.did, ev.commitCID.String(), prevData, sinceRev, ev.rev, opsJSON, carSlice).Scan(&seq); err != nil { + return 0, fmt.Errorf("repo: append firehose event for %s: %w", ev.did, err) + } + return seq, nil +} + +// writeCARSlice serializes blocks as a CARv1 stream rooted at the commit +// CID — exactly the `blocks` payload of a subscribeRepos #commit frame. +// The commit (root) block is written first — atproto CAR consumers +// conventionally expect the root at the front — followed by the remaining +// blocks in their original deterministic order. +func writeCARSlice(root cid.Cid, blks []blockformat.Block) ([]byte, error) { + var buf bytes.Buffer + if err := car.WriteHeader(&car.CarHeader{ + Roots: []cid.Cid{root}, + Version: 1, + }, &buf); err != nil { + return nil, fmt.Errorf("repo: write CAR header: %w", err) + } + for _, blk := range blks { + if blk.Cid().Equals(root) { + if err := carutil.LdWrite(&buf, blk.Cid().Bytes(), blk.RawData()); err != nil { + return nil, fmt.Errorf("repo: write CAR block %s: %w", blk.Cid(), err) + } + } + } + for _, blk := range blks { + if blk.Cid().Equals(root) { + continue + } + if err := carutil.LdWrite(&buf, blk.Cid().Bytes(), blk.RawData()); err != nil { + return nil, fmt.Errorf("repo: write CAR block %s: %w", blk.Cid(), err) + } + } + return buf.Bytes(), nil +} + +// ExportCAR writes the DID's full repo as a CARv1 stream rooted at the +// current head commit. NOTE (task 04): until block garbage collection +// exists, this includes every historical block for the DID, not just the +// reachable set — harmless for CAR readers (they traverse from the root) +// but larger than a minimal export. +func (m *Manager) ExportCAR(ctx context.Context, did string) ([]byte, error) { + tx, err := m.db.BeginTx(ctx, &sql.TxOptions{ReadOnly: true}) + if err != nil { + return nil, fmt.Errorf("repo: begin read tx: %w", err) + } + defer func() { _ = tx.Rollback() }() + + state, err := readRepoState(ctx, tx, did, false) + if err != nil { + return nil, err + } + head, err := cid.Parse(state.headCID) + if err != nil { + return nil, fmt.Errorf("repo: parse head cid %q for %s: %w", state.headCID, did, err) + } + + var buf bytes.Buffer + if err := car.WriteHeader(&car.CarHeader{Roots: []cid.Cid{head}, Version: 1}, &buf); err != nil { + return nil, fmt.Errorf("repo: write CAR header: %w", err) + } + + // Head commit block first (readers conventionally expect the root + // early), then everything else. + src := &txBlockSource{tx: tx, did: did} + headBlk, err := src.Get(ctx, head) + if err != nil { + return nil, fmt.Errorf("repo: read head commit %s for %s: %w", state.headCID, did, err) + } + if err := carutil.LdWrite(&buf, headBlk.Cid().Bytes(), headBlk.RawData()); err != nil { + return nil, fmt.Errorf("repo: write CAR block %s: %w", head, err) + } + + rows, err := tx.QueryContext(ctx, + `SELECT cid, bytes FROM blocks WHERE did = $1 AND cid <> $2 ORDER BY created_at, cid`, + did, state.headCID) + if err != nil { + return nil, fmt.Errorf("repo: list blocks for %s: %w", did, err) + } + defer rows.Close() + for rows.Next() { + var cidStr string + var raw []byte + if err := rows.Scan(&cidStr, &raw); err != nil { + return nil, fmt.Errorf("repo: scan block for %s: %w", did, err) + } + c, err := cid.Parse(cidStr) + if err != nil { + return nil, fmt.Errorf("repo: parse stored cid %q for %s: %w", cidStr, did, err) + } + if err := carutil.LdWrite(&buf, c.Bytes(), raw); err != nil { + return nil, fmt.Errorf("repo: write CAR block %s: %w", c, err) + } + } + if err := rows.Err(); err != nil { + return nil, fmt.Errorf("repo: iterate blocks for %s: %w", did, err) + } + return buf.Bytes(), nil +} diff --git a/internal/repo/repo.go b/internal/repo/repo.go new file mode 100644 index 0000000..890e50f --- /dev/null +++ b/internal/repo/repo.go @@ -0,0 +1,509 @@ +// Package repo maintains a real, signed atproto repository (Merkle Search +// Tree + commit chain) per bridged DID, persisted in postgres. It is the Go +// port of arroba's virtual-PDS job, built on indigo's atproto/repo and +// atproto/repo/mst primitives: every write produces a properly signed v3 +// commit with a monotonically increasing rev TID, stores the new blocks, +// and appends a firehose event in the same transaction (task 04 serves +// those events over com.atproto.sync.subscribeRepos). +package repo + +import ( + "bytes" + "context" + "database/sql" + stderrors "errors" + "fmt" + "log/slog" + "sync" + + indigorepo "github.com/bluesky-social/indigo/atproto/repo" + "github.com/bluesky-social/indigo/atproto/repo/mst" + + "github.com/bluesky-social/indigo/atproto/atcrypto" + "github.com/bluesky-social/indigo/atproto/atdata" + "github.com/bluesky-social/indigo/atproto/syntax" + + blockformat "github.com/ipfs/go-block-format" + "github.com/ipfs/go-cid" + + "tidepool/internal/errors" +) + +// KeyUse says what a signing key is being requested for, so key custody +// can apply consent policy per operation kind: tombstoned (consent-revoked) +// actors are frozen for new writes but their records must remain deletable. +type KeyUse int + +const ( + // KeyUseWrite covers record creates and updates. + KeyUseWrite KeyUse = iota + // KeyUseDelete covers record deletions — including scrubbing a + // tombstoned actor's records, which must always be possible (that IS + // the intent of consent revocation). + KeyUseDelete +) + +// SigningKeys resolves the commit-signing key for a bridged DID. +// identity.ActorKeys implements it: tombstoned actors' keys are never +// released for KeyUseWrite (errors.IsTombstoned), which is what freezes +// their repos, but KeyUseDelete still releases the key so task 05's +// Delete(Actor) → scrub-records flow works after a consent flip. +type SigningKeys interface { + SigningKey(ctx context.Context, did string, use KeyUse) (atcrypto.PrivateKey, error) +} + +// commitAdvisoryLockKey is the postgres advisory-lock key every commit +// transaction takes (pg_advisory_xact_lock) before reading repo_state. It +// is ONE GLOBAL lock — not per-DID — on purpose, buying two guarantees: +// +// 1. Genesis safety across processes: SELECT ... FOR UPDATE on a missing +// repo_state row locks nothing, so without this two processes could +// both build a genesis commit for the same DID and the head upsert +// would silently overwrite one of them. +// 2. Firehose cursor safety: firehose_events.seq (bigserial) is assigned +// at INSERT time, but rows become visible in transaction-commit order. +// Serializing all commits makes seq order equal commit-visibility +// order, so a task-04 tailer doing `WHERE seq > cursor` can never +// permanently skip an event. +// +// Globally serializing commits is acceptable at bridge write volume +// (PLAN.md: one deployment, full control of emission order). The per-DID +// mutex and the repo_state row lock are kept as fast-path/backstop. +// +// This key MUST stay distinct from internal/testutil's cross-package test +// lock key (0x7469646570, session-scoped, held for a whole test binary) — +// sharing it would deadlock every commit made from tests. +const commitAdvisoryLockKey int64 = 0x7469646570636d // "tidep"+"cm" (commit) + +// Manager owns all bridged repos. Writes take a per-DID in-process mutex +// (local fairness), then a global advisory transaction lock +// (commitAdvisoryLockKey) that serializes every commit across processes — +// that lock, not the repo_state row lock, is what makes genesis commits +// race-free and firehose seq order match commit-visibility order. The row +// lock (SELECT ... FOR UPDATE) is kept as a backstop. +type Manager struct { + db *sql.DB + keys SigningKeys + logger *slog.Logger + + mu sync.Mutex + // locks is never evicted: it is bounded by the number of bridged + // actors, and a stale mutex per DID is 8 bytes of pointer. + locks map[string]*sync.Mutex +} + +// NewManager builds the repo manager. db and keys must be non-nil; a nil +// logger falls back to slog.Default(). +func NewManager(db *sql.DB, keys SigningKeys, logger *slog.Logger) (*Manager, error) { + if db == nil { + return nil, errors.NewValidationError("db", "must not be nil") + } + if keys == nil { + return nil, errors.NewValidationError("keys", "must not be nil") + } + if logger == nil { + logger = slog.Default() + } + return &Manager{ + db: db, + keys: keys, + logger: logger, + locks: make(map[string]*sync.Mutex), + }, nil +} + +// CommitResult reports what a successful PutRecord/DeleteRecord did. +// Tasks 04/05 consume it to build subscribeRepos frames and ap_objects +// mappings without re-querying. +type CommitResult struct { + // RecordCID is the CID of the written record; empty for deletes. + RecordCID string + // CommitCID is the signed commit block's CID — the repo head after + // this write (for a NoOp re-put, the pre-existing head). + CommitCID string + // Rev is the commit's rev TID (the pre-existing rev on NoOp). + Rev string + // Seq is the firehose_events cursor assigned to this commit's event. + // Zero on NoOp: no event was emitted. + Seq int64 + // NoOp marks the idempotent re-put path: the identical record already + // existed, so no new commit or firehose event was produced. + NoOp bool +} + +// PutRecord creates or updates a record and commits the change. The first +// write to a DID creates its repo (genesis commit). Re-putting an identical +// record is an idempotent no-op: no new commit or firehose event is +// emitted, and the result carries NoOp with the existing CID, head, and rev +// (deterministic rkeys make re-ingestion hit this path on purpose). +func (m *Manager) PutRecord(ctx context.Context, did, collection, rkey string, record map[string]any) (*CommitResult, error) { + if err := validateRecord(record); err != nil { + return nil, err + } + recordBytes, err := atdata.MarshalCBOR(record) + if err != nil { + return nil, fmt.Errorf("repo: encode record: %w", err) + } + c, err := cidForBlock(recordBytes) + if err != nil { + return nil, err + } + return m.commitWrite(ctx, did, collection, rkey, &c, recordBytes) +} + +// DeleteRecord removes a record and commits the change. A missing record — +// or a repo that does not exist yet — is an error satisfying +// errors.IsNotFound. The result's RecordCID is empty. +func (m *Manager) DeleteRecord(ctx context.Context, did, collection, rkey string) (*CommitResult, error) { + return m.commitWrite(ctx, did, collection, rkey, nil, nil) +} + +// GetRecord reads the current version of a record. Missing repo or record +// is an error satisfying errors.IsNotFound. +func (m *Manager) GetRecord(ctx context.Context, did, collection, rkey string) (record map[string]any, recordCID string, err error) { + path, _, err := validatePath(did, collection, rkey) + if err != nil { + return nil, "", err + } + + // Note this tx runs READ COMMITTED, i.e. per-statement snapshots — it + // does NOT freeze one snapshot across the reads below. Consistency + // actually rests on blocks being content-addressed and append-only: + // once the head pointer is read, every block it references is immutable + // and present. Future block GC must preserve that property for any head + // a reader may still hold. + tx, err := m.db.BeginTx(ctx, &sql.TxOptions{ReadOnly: true}) + if err != nil { + return nil, "", fmt.Errorf("repo: begin read tx: %w", err) + } + defer func() { _ = tx.Rollback() }() + + state, err := readRepoState(ctx, tx, did, false) + if err != nil { + return nil, "", err + } + tree, _, err := loadTree(ctx, tx, did, state.headCID) + if err != nil { + return nil, "", err + } + valCID, err := tree.Get([]byte(path)) + if err != nil { + return nil, "", fmt.Errorf("repo: MST get %s: %w", path, err) + } + if valCID == nil { + return nil, "", errors.NewNotFoundError("record", fmt.Sprintf("at://%s/%s", did, path)) + } + src := &txBlockSource{tx: tx, did: did} + blk, err := src.Get(ctx, *valCID) + if err != nil { + return nil, "", fmt.Errorf("repo: read record block %s: %w", valCID, err) + } + record, err = atdata.UnmarshalCBOR(blk.RawData()) + if err != nil { + return nil, "", fmt.Errorf("repo: decode record %s: %w", valCID, err) + } + return record, valCID.String(), nil +} + +// Head returns the current head commit CID and rev for a DID. A repo that +// has never committed satisfies errors.IsNotFound. +func (m *Manager) Head(ctx context.Context, did string) (headCID string, rev string, err error) { + state, err := m.readState(ctx, did) + if err != nil { + return "", "", err + } + return state.headCID, state.rev, nil +} + +type repoState struct { + headCID string + rev string +} + +func (m *Manager) readState(ctx context.Context, did string) (*repoState, error) { + var st repoState + err := m.db.QueryRowContext(ctx, + `SELECT head_cid, rev FROM repo_state WHERE did = $1`, did).Scan(&st.headCID, &st.rev) + if stderrors.Is(err, sql.ErrNoRows) { + return nil, errors.NewNotFoundError("repo", did) + } + if err != nil { + return nil, fmt.Errorf("repo: read repo_state for %s: %w", did, err) + } + return &st, nil +} + +// commitWrite is the single write path: PutRecord passes the new record CID +// and bytes, DeleteRecord passes nil. It serializes on the per-DID mutex +// and the global commit advisory lock, applies the mutation to the MST, +// signs a new commit, and persists blocks, head, and the firehose event in +// one transaction. +func (m *Manager) commitWrite(ctx context.Context, did, collection, rkey string, newCID *cid.Cid, recordBytes []byte) (*CommitResult, error) { + path, parsedDID, err := validatePath(did, collection, rkey) + if err != nil { + return nil, err + } + use := KeyUseWrite + if newCID == nil { + use = KeyUseDelete + } + + lock := m.lockFor(did) + lock.Lock() + defer lock.Unlock() + + tx, err := m.db.BeginTx(ctx, nil) + if err != nil { + return nil, fmt.Errorf("repo: begin tx: %w", err) + } + defer func() { _ = tx.Rollback() }() + + // Global commit serialization; see commitAdvisoryLockKey. Released + // automatically when the transaction commits or rolls back. + if _, err := tx.ExecContext(ctx, + `SELECT pg_advisory_xact_lock($1)`, commitAdvisoryLockKey); err != nil { + return nil, fmt.Errorf("repo: take commit advisory lock: %w", err) + } + + // The signing-key fetch is also the consent gate: tombstoned actors get + // no key for writes (their repos are frozen), while deletes still get + // one (scrubbing must survive a consent flip). It runs with the locks + // held to narrow the TOCTOU against a concurrent consent change — but a + // flip racing an in-flight commit still has a residual one-commit + // window, because the consent row is read outside this transaction. + // Full elimination needs the consent read inside the same tx (deferred). + signingKey, err := m.keys.SigningKey(ctx, did, use) + if err != nil { + return nil, err + } + + state, err := readRepoState(ctx, tx, did, true) + if err != nil && !errors.IsNotFound(err) { + return nil, err + } + + var tree *mst.Tree + var prevRev string + var prevData *cid.Cid // MST root before this commit (firehose prevData) + if state == nil { + if newCID == nil { + return nil, errors.NewNotFoundError("record", fmt.Sprintf("at://%s/%s", did, path)) + } + empty := mst.NewEmptyTree() + tree = &empty + } else { + prevRev = state.rev + var headData cid.Cid + tree, headData, err = loadTree(ctx, tx, did, state.headCID) + if err != nil { + return nil, err + } + prevData = &headData + } + + // Note: indigo's mst.Tree.Remove returns (nil, nil) for a missing key — + // NOT an error — so any ApplyOp error here is real corruption + // (mst.ErrPartialTree etc.) and must surface as an internal error. + // Missing-record deletes are detected by op.Prev == nil below. + op, err := indigorepo.ApplyOp(tree, path, newCID) + if err != nil { + return nil, fmt.Errorf("repo: apply op %s: %w", path, err) + } + if newCID == nil && op.Prev == nil { + return nil, errors.NewNotFoundError("record", fmt.Sprintf("at://%s/%s", did, path)) + } + if newCID != nil && op.Prev != nil && op.Prev.Equals(*newCID) { + // Identical re-put: idempotent no-op, keep the existing commit. + // op.Prev != nil implies the repo exists, so state is non-nil here. + return &CommitResult{ + RecordCID: newCID.String(), + CommitCID: state.headCID, + Rev: prevRev, + NoOp: true, + }, nil + } + + // New blocks this commit introduces: MST diff nodes + the record block + // (for puts) + the commit block, captured in order for the CAR slice. + newBlocks := newMemBlockstore() + newRoot, err := tree.WriteDiffBlocks(ctx, newBlocks) + if err != nil { + return nil, fmt.Errorf("repo: write MST diff: %w", err) + } + + if newCID != nil { + blk, err := blockformat.NewBlockWithCid(recordBytes, *newCID) + if err != nil { + return nil, fmt.Errorf("repo: build record block: %w", err) + } + if err := newBlocks.Put(ctx, blk); err != nil { + return nil, err + } + } + + rev, err := NextRev(prevRev) + if err != nil { + return nil, fmt.Errorf("repo: next rev after %q for %s: %w", prevRev, did, err) + } + + commit := indigorepo.Commit{ + DID: parsedDID.String(), + Version: indigorepo.ATPROTO_REPO_VERSION, + Prev: nil, // v3 commits carry no prev pointer; prevData rides the firehose event + Data: *newRoot, + Rev: rev.String(), + } + if err := commit.Sign(signingKey); err != nil { + return nil, fmt.Errorf("repo: sign commit for %s: %w", did, err) + } + var commitBuf bytes.Buffer + if err := commit.MarshalCBOR(&commitBuf); err != nil { + return nil, fmt.Errorf("repo: encode commit: %w", err) + } + commitCID, err := cidForBlock(commitBuf.Bytes()) + if err != nil { + return nil, err + } + commitBlock, err := blockformat.NewBlockWithCid(commitBuf.Bytes(), commitCID) + if err != nil { + return nil, fmt.Errorf("repo: build commit block: %w", err) + } + if err := newBlocks.Put(ctx, commitBlock); err != nil { + return nil, err + } + + for _, blk := range newBlocks.ordered() { + if _, err := tx.ExecContext(ctx, + `INSERT INTO blocks (did, cid, bytes) VALUES ($1, $2, $3) ON CONFLICT (did, cid) DO NOTHING`, + did, blk.Cid().String(), blk.RawData()); err != nil { + return nil, fmt.Errorf("repo: store block %s: %w", blk.Cid(), err) + } + } + if _, err := tx.ExecContext(ctx, ` + INSERT INTO repo_state (did, head_cid, rev) VALUES ($1, $2, $3) + ON CONFLICT (did) DO UPDATE SET + head_cid = EXCLUDED.head_cid, + rev = EXCLUDED.rev, + updated_at = CURRENT_TIMESTAMP`, + did, commitCID.String(), rev.String()); err != nil { + return nil, fmt.Errorf("repo: update repo_state for %s: %w", did, err) + } + + // The firehose event rides the same transaction: a commit either + // appears on the stream exactly once or does not exist at all. + seq, err := appendFirehoseEvent(ctx, tx, firehoseEvent{ + did: did, + commitCID: commitCID, + prevData: prevData, + sinceRev: prevRev, // empty on genesis → NULL + rev: rev.String(), + ops: []Op{opFromIndigoOp(op)}, + blocks: newBlocks.ordered(), + }) + if err != nil { + return nil, err + } + + if err := tx.Commit(); err != nil { + return nil, fmt.Errorf("repo: commit tx for %s: %w", did, err) + } + + m.logger.Debug("repo commit", + "did", did, "rev", rev.String(), "commit", commitCID.String(), "path", path, "seq", seq) + res := &CommitResult{ + CommitCID: commitCID.String(), + Rev: rev.String(), + Seq: seq, + } + if newCID != nil { + res.RecordCID = newCID.String() + } + return res, nil +} + +// lockFor returns the per-DID write mutex, creating it on first use. +func (m *Manager) lockFor(did string) *sync.Mutex { + m.mu.Lock() + defer m.mu.Unlock() + lock, ok := m.locks[did] + if !ok { + lock = &sync.Mutex{} + m.locks[did] = lock + } + return lock +} + +// readRepoState reads the head pointer, taking the row lock when forUpdate +// (write path). The row lock is only a backstop: cross-process commit +// serialization comes from the global advisory lock (commitAdvisoryLockKey) +// — FOR UPDATE on a missing row locks nothing, so it cannot protect genesis +// commits. A repo with no commits yet satisfies errors.IsNotFound. +func readRepoState(ctx context.Context, tx *sql.Tx, did string, forUpdate bool) (*repoState, error) { + query := `SELECT head_cid, rev FROM repo_state WHERE did = $1` + if forUpdate { + query += ` FOR UPDATE` + } + var st repoState + err := tx.QueryRowContext(ctx, query, did).Scan(&st.headCID, &st.rev) + if stderrors.Is(err, sql.ErrNoRows) { + return nil, errors.NewNotFoundError("repo", did) + } + if err != nil { + return nil, fmt.Errorf("repo: read repo_state for %s: %w", did, err) + } + return &st, nil +} + +// loadTree loads the full MST behind a head commit from the DID's blocks. +// It returns the tree and the commit's data (MST root) CID. +func loadTree(ctx context.Context, tx *sql.Tx, did, headCID string) (*mst.Tree, cid.Cid, error) { + head, err := cid.Parse(headCID) + if err != nil { + return nil, cid.Undef, fmt.Errorf("repo: parse head cid %q for %s: %w", headCID, did, err) + } + src := &txBlockSource{tx: tx, did: did} + blk, err := src.Get(ctx, head) + if err != nil { + return nil, cid.Undef, fmt.Errorf("repo: read head commit %s for %s: %w", headCID, did, err) + } + var commit indigorepo.Commit + if err := commit.UnmarshalCBOR(bytes.NewReader(blk.RawData())); err != nil { + return nil, cid.Undef, fmt.Errorf("repo: decode head commit %s for %s: %w", headCID, did, err) + } + tree, err := mst.LoadTreeFromStore(ctx, src, commit.Data) + if err != nil { + return nil, cid.Undef, fmt.Errorf("repo: load MST for %s: %w", did, err) + } + return tree, commit.Data, nil +} + +// validatePath validates the identifier triple and returns the MST path. +func validatePath(did, collection, rkey string) (string, syntax.DID, error) { + parsedDID, err := syntax.ParseDID(did) + if err != nil { + return "", "", errors.NewValidationError("did", err.Error()) + } + nsid, err := syntax.ParseNSID(collection) + if err != nil { + return "", "", errors.NewValidationError("collection", err.Error()) + } + parsedRKey, err := syntax.ParseRecordKey(rkey) + if err != nil { + return "", "", errors.NewValidationError("rkey", err.Error()) + } + return nsid.String() + "/" + parsedRKey.String(), parsedDID, nil +} + +// validateRecord applies the checks record maps must pass before encoding: +// non-nil and carrying the $type every atproto record requires. Full data +// model validation happens implicitly in atdata.MarshalCBOR. +func validateRecord(record map[string]any) error { + if record == nil { + return errors.NewValidationError("record", "must not be nil") + } + t, ok := record["$type"].(string) + if !ok || t == "" { + return errors.NewValidationError("record", "must carry a non-empty $type string") + } + return nil +} diff --git a/internal/repo/repo_test.go b/internal/repo/repo_test.go new file mode 100644 index 0000000..b339f2f --- /dev/null +++ b/internal/repo/repo_test.go @@ -0,0 +1,622 @@ +package repo + +import ( + "bytes" + "context" + "database/sql" + "encoding/json" + "fmt" + "sync" + "testing" + "time" + + indigorepo "github.com/bluesky-social/indigo/atproto/repo" + + "github.com/bluesky-social/indigo/atproto/atcrypto" + "github.com/bluesky-social/indigo/atproto/syntax" + car "github.com/ipld/go-car" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "tidepool/internal/errors" + "tidepool/internal/testutil" +) + +const ( + testDID = "did:plc:ewvi7nxzyoun6zhxrhs64oiz" + testOtherDID = "did:plc:44ybard66vv44zksje25o7dz" + testCollection = "social.coves.community.post" +) + +// staticKeys signs every DID with one fixed key — the repo layer only needs +// "give me the signing key for this DID", so tests stay decoupled from the +// identity package (which has its own ActorKeys tests). It records the +// KeyUse of every request so tests can assert the consent-gate routing. +type staticKeys struct { + key *atcrypto.PrivateKeyK256 + + mu sync.Mutex + uses []KeyUse +} + +func (s *staticKeys) SigningKey(_ context.Context, _ string, use KeyUse) (atcrypto.PrivateKey, error) { + s.mu.Lock() + s.uses = append(s.uses, use) + s.mu.Unlock() + return s.key, nil +} + +func (s *staticKeys) recordedUses() []KeyUse { + s.mu.Lock() + defer s.mu.Unlock() + return append([]KeyUse(nil), s.uses...) +} + +func testManager(t *testing.T) (*Manager, *sql.DB, *atcrypto.PrivateKeyK256) { + t.Helper() + manager, database, key, _ := testManagerWithKeys(t) + return manager, database, key +} + +func testManagerWithKeys(t *testing.T) (*Manager, *sql.DB, *atcrypto.PrivateKeyK256, *staticKeys) { + t.Helper() + database := testutil.DB(t) + testutil.Truncate(t, database, "blocks", "repo_state", "firehose_events") + key, err := atcrypto.GeneratePrivateKeyK256() + require.NoError(t, err) + keys := &staticKeys{key: key} + manager, err := NewManager(database, keys, nil) + require.NoError(t, err) + return manager, database, key, keys +} + +func testRecord(text string) map[string]any { + return map[string]any{ + "$type": testCollection, + "text": text, + "createdAt": "2026-05-01T12:00:00Z", + } +} + +func testRKey(i int) string { + published := time.Date(2026, 5, 1, 12, 0, i, 0, time.UTC) + tid, err := DeterministicTID(published, fmt.Sprintf("https://lemmy.world/post/%d", i)) + if err != nil { + panic(err) + } + return tid.String() +} + +func TestPutRecord_RoundTripThroughIndigoCAR(t *testing.T) { + manager, _, key := testManager(t) + ctx := t.Context() + + const n = 25 + wantCIDs := make(map[string]string, n) // path -> record CID + var lastSeq int64 + for i := 0; i < n; i++ { + rkey := testRKey(i) + res, err := manager.PutRecord(ctx, testDID, testCollection, rkey, testRecord(fmt.Sprintf("post %d", i))) + require.NoError(t, err) + require.NotEmpty(t, res.RecordCID) + require.NotEmpty(t, res.CommitCID) + require.NotEmpty(t, res.Rev) + require.False(t, res.NoOp) + require.Greater(t, res.Seq, lastSeq, "seq must strictly increase per commit") + lastSeq = res.Seq + wantCIDs[testCollection+"/"+rkey] = res.RecordCID + } + + // Read one back through the manager. + record, gotCID, err := manager.GetRecord(ctx, testDID, testCollection, testRKey(7)) + require.NoError(t, err) + assert.Equal(t, "post 7", record["text"]) + assert.Equal(t, wantCIDs[testCollection+"/"+testRKey(7)], gotCID) + + // Export the whole repo as CAR and reload it with indigo. + carBytes, err := manager.ExportCAR(ctx, testDID) + require.NoError(t, err) + commit, loaded, err := indigorepo.LoadRepoFromCAR(ctx, bytes.NewReader(carBytes)) + require.NoError(t, err, "indigo must accept our CAR export") + + assert.Equal(t, testDID, commit.DID) + require.NoError(t, commit.VerifyStructure()) + + // The signature must verify with the signing key's public half. + pub, err := key.PublicKey() + require.NoError(t, err) + require.NoError(t, commit.VerifySignature(pub), "commit signature must verify with the minted key") + + // The MST root recomputed from the loaded tree must match the signed + // commit's data CID. + root, err := loaded.MST.RootCID() + require.NoError(t, err) + assert.Equal(t, commit.Data.String(), root.String(), "recomputed MST root must match the signed commit") + + // Every record must be present with the CID we returned at write time. + for path, want := range wantCIDs { + nsid, rkey, err := syntax.ParseRepoPath(path) + require.NoError(t, err) + _, c, err := loaded.GetRecordBytes(ctx, nsid, rkey) + require.NoError(t, err, "record %s must be in the reloaded repo", path) + assert.Equal(t, want, c.String()) + } + + // Head bookkeeping matches the export. + headCID, headRev, err := manager.Head(ctx, testDID) + require.NoError(t, err) + assert.Equal(t, commit.Rev, headRev) + assert.NotEmpty(t, headCID) +} + +func TestPutRecord_UpdateAndDelete(t *testing.T) { + manager, _, _ := testManager(t) + ctx := t.Context() + rkey := testRKey(1) + + first, err := manager.PutRecord(ctx, testDID, testCollection, rkey, testRecord("v1")) + require.NoError(t, err) + + second, err := manager.PutRecord(ctx, testDID, testCollection, rkey, testRecord("v2")) + require.NoError(t, err) + assert.NotEqual(t, first.RecordCID, second.RecordCID) + assert.NotEqual(t, first.CommitCID, second.CommitCID) + assert.Greater(t, second.Rev, first.Rev, "revs must be monotonic") + assert.Greater(t, second.Seq, first.Seq, "seq must be monotonic") + + record, gotCID, err := manager.GetRecord(ctx, testDID, testCollection, rkey) + require.NoError(t, err) + assert.Equal(t, "v2", record["text"]) + assert.Equal(t, second.RecordCID, gotCID) + + deleted, err := manager.DeleteRecord(ctx, testDID, testCollection, rkey) + require.NoError(t, err) + assert.Greater(t, deleted.Rev, second.Rev) + assert.Empty(t, deleted.RecordCID, "deletes carry no record CID") + assert.False(t, deleted.NoOp) + + _, _, err = manager.GetRecord(ctx, testDID, testCollection, rkey) + assert.True(t, errors.IsNotFound(err), "deleted record must read as not found") + + // Deleting again: gone. + _, err = manager.DeleteRecord(ctx, testDID, testCollection, rkey) + assert.True(t, errors.IsNotFound(err)) +} + +func TestPutRecord_IdenticalRePutIsNoOp(t *testing.T) { + manager, database, _ := testManager(t) + ctx := t.Context() + rkey := testRKey(2) + + first, err := manager.PutRecord(ctx, testDID, testCollection, rkey, testRecord("same")) + require.NoError(t, err) + require.False(t, first.NoOp) + + second, err := manager.PutRecord(ctx, testDID, testCollection, rkey, testRecord("same")) + require.NoError(t, err) + assert.Equal(t, first.RecordCID, second.RecordCID, "identical content must produce the identical CID") + assert.Equal(t, first.Rev, second.Rev, "no new commit for an identical re-put") + assert.Equal(t, first.CommitCID, second.CommitCID, "no-op re-put reports the existing head") + assert.True(t, second.NoOp, "identical re-put must be marked NoOp") + assert.Zero(t, second.Seq, "no firehose event, no seq") + + var events int + require.NoError(t, database.QueryRowContext(ctx, + `SELECT COUNT(*) FROM firehose_events WHERE did = $1`, testDID).Scan(&events)) + assert.Equal(t, 1, events, "idempotent re-put must not emit a firehose event") +} + +func TestNewManager_ValidatesInputs(t *testing.T) { + database := testutil.DB(t) + key, err := atcrypto.GeneratePrivateKeyK256() + require.NoError(t, err) + + _, err = NewManager(nil, &staticKeys{key: key}, nil) + assert.True(t, errors.IsValidation(err), "nil db must be rejected") + + _, err = NewManager(database, nil, nil) + assert.True(t, errors.IsValidation(err), "nil keys must be rejected") +} + +func TestCommitWrite_KeyUseRouting(t *testing.T) { + manager, _, _, keys := testManagerWithKeys(t) + ctx := t.Context() + rkey := testRKey(8) + + _, err := manager.PutRecord(ctx, testDID, testCollection, rkey, testRecord("v1")) + require.NoError(t, err) + _, err = manager.PutRecord(ctx, testDID, testCollection, rkey, testRecord("v2")) + require.NoError(t, err) + _, err = manager.DeleteRecord(ctx, testDID, testCollection, rkey) + require.NoError(t, err) + + assert.Equal(t, []KeyUse{KeyUseWrite, KeyUseWrite, KeyUseDelete}, keys.recordedUses(), + "puts must request KeyUseWrite, deletes KeyUseDelete (tombstone scrub depends on it)") +} + +func TestDeleteRecord_MissingRepoOrRecord(t *testing.T) { + manager, _, _ := testManager(t) + ctx := t.Context() + + _, err := manager.DeleteRecord(ctx, testDID, testCollection, testRKey(3)) + assert.True(t, errors.IsNotFound(err), "delete on a repo that does not exist yet") + + _, _, err = manager.GetRecord(ctx, testDID, testCollection, testRKey(3)) + assert.True(t, errors.IsNotFound(err)) + + _, err = manager.PutRecord(ctx, testDID, testCollection, testRKey(3), testRecord("x")) + require.NoError(t, err) + _, err = manager.DeleteRecord(ctx, testDID, testCollection, testRKey(4)) + assert.True(t, errors.IsNotFound(err), "delete of a record that was never written") +} + +func TestPutRecord_ValidatesInputs(t *testing.T) { + manager, _, _ := testManager(t) + ctx := t.Context() + + _, err := manager.PutRecord(ctx, "not-a-did", testCollection, testRKey(0), testRecord("x")) + assert.True(t, errors.IsValidation(err)) + + _, err = manager.PutRecord(ctx, testDID, "NotAnNSID!!", testRKey(0), testRecord("x")) + assert.True(t, errors.IsValidation(err)) + + _, err = manager.PutRecord(ctx, testDID, testCollection, "bad rkey!", testRecord("x")) + assert.True(t, errors.IsValidation(err)) + + _, err = manager.PutRecord(ctx, testDID, testCollection, testRKey(0), nil) + assert.True(t, errors.IsValidation(err)) + + _, err = manager.PutRecord(ctx, testDID, testCollection, testRKey(0), map[string]any{"text": "no type"}) + assert.True(t, errors.IsValidation(err), "records must carry $type") +} + +func TestFirehoseEvents_AtomicWithCommits(t *testing.T) { + manager, database, _ := testManager(t) + ctx := t.Context() + + rkey := testRKey(5) + res1, err := manager.PutRecord(ctx, testDID, testCollection, rkey, testRecord("v1")) + require.NoError(t, err) + res2, err := manager.PutRecord(ctx, testDID, testCollection, rkey, testRecord("v2")) + require.NoError(t, err) + res3, err := manager.DeleteRecord(ctx, testDID, testCollection, rkey) + require.NoError(t, err) + cid1, rev1, rev2, rev3 := res1.RecordCID, res1.Rev, res2.Rev, res3.Rev + + rows, err := database.QueryContext(ctx, ` + SELECT seq, commit_cid, prev_data_cid, since_rev, rev, ops, car + FROM firehose_events WHERE did = $1 ORDER BY seq`, testDID) + require.NoError(t, err) + defer rows.Close() + + type event struct { + seq int64 + commitCID string + prevData *string + sinceRev *string + rev string + ops []Op + car []byte + } + var events []event + for rows.Next() { + var ev event + var opsJSON []byte + require.NoError(t, rows.Scan(&ev.seq, &ev.commitCID, &ev.prevData, &ev.sinceRev, &ev.rev, &opsJSON, &ev.car)) + require.NoError(t, json.Unmarshal(opsJSON, &ev.ops)) + events = append(events, ev) + } + require.NoError(t, rows.Err()) + require.Len(t, events, 3, "every commit appends exactly one event") + + // seq strictly increasing, matching the CommitResults; revs match too. + assert.Less(t, events[0].seq, events[1].seq) + assert.Less(t, events[1].seq, events[2].seq) + assert.Equal(t, []int64{res1.Seq, res2.Seq, res3.Seq}, + []int64{events[0].seq, events[1].seq, events[2].seq}, + "CommitResult.Seq must be the stored firehose cursor") + assert.Equal(t, []string{rev1, rev2, rev3}, + []string{events[0].rev, events[1].rev, events[2].rev}) + assert.Equal(t, []string{res1.CommitCID, res2.CommitCID, res3.CommitCID}, + []string{events[0].commitCID, events[1].commitCID, events[2].commitCID}) + + // Ops: create → update (with prev) → delete (with prev). + require.Len(t, events[0].ops, 1) + assert.Equal(t, Op{Action: OpActionCreate, Path: testCollection + "/" + rkey, CID: cid1}, events[0].ops[0]) + assert.Equal(t, OpActionUpdate, events[1].ops[0].Action) + assert.Equal(t, cid1, events[1].ops[0].Prev, "update op must carry the previous record CID (sync v1.1)") + assert.Equal(t, OpActionDelete, events[2].ops[0].Action) + assert.Empty(t, events[2].ops[0].CID) + assert.NotEmpty(t, events[2].ops[0].Prev) + + // prevData: null on genesis, then the prior commit's MST root. + assert.Nil(t, events[0].prevData, "genesis event has no prevData") + require.NotNil(t, events[1].prevData) + + // since_rev: null on genesis, then the previous commit's rev (the + // subscribeRepos #commit `since` field task 04 serves). + assert.Nil(t, events[0].sinceRev, "genesis event has no since_rev") + require.NotNil(t, events[1].sinceRev) + assert.Equal(t, rev1, *events[1].sinceRev) + require.NotNil(t, events[2].sinceRev) + assert.Equal(t, rev2, *events[2].sinceRev) + + // The genesis CAR slice is a complete mini-repo: indigo can load it. + commit0, loaded0, err := indigorepo.LoadRepoFromCAR(ctx, bytes.NewReader(events[0].car)) + require.NoError(t, err, "genesis CAR slice must parse as a CAR with the commit as root") + assert.Equal(t, events[0].commitCID, mustRootCID(t, events[0].car)) + assert.Equal(t, rev1, commit0.Rev) + nsid, rk, err := syntax.ParseRepoPath(testCollection + "/" + rkey) + require.NoError(t, err) + _, c, err := loaded0.GetRecordBytes(ctx, nsid, rk) + require.NoError(t, err) + assert.Equal(t, cid1, c.String()) + + // Every event's CAR slice has the commit block as its root, and the + // commit block is physically FIRST in the stream (atproto consumers + // conventionally expect root-first). + for _, ev := range events { + commit, _, err := indigorepo.LoadCommitFromCAR(ctx, bytes.NewReader(ev.car)) + require.NoError(t, err, "event %d CAR slice must contain its commit block", ev.seq) + assert.Equal(t, ev.rev, commit.Rev) + assert.Equal(t, ev.commitCID, mustRootCID(t, ev.car)) + + reader, err := car.NewCarReader(bytes.NewReader(ev.car)) + require.NoError(t, err) + firstBlk, err := reader.Next() + require.NoError(t, err) + assert.Equal(t, ev.commitCID, firstBlk.Cid().String(), + "commit (root) block must be written first in the CAR slice") + } + + // prevData of event 2 equals the MST root of commit 1. + commit1, _, err := indigorepo.LoadCommitFromCAR(ctx, bytes.NewReader(events[1].car)) + require.NoError(t, err) + assert.Equal(t, commit1.Data.String(), *events[2].prevData) +} + +func mustRootCID(t *testing.T, carBytes []byte) string { + t.Helper() + _, root, err := indigorepo.LoadCommitFromCAR(context.Background(), bytes.NewReader(carBytes)) + require.NoError(t, err) + return root.String() +} + +func TestPutRecord_ConcurrentWritesSerialize(t *testing.T) { + manager, database, _ := testManager(t) + ctx := t.Context() + + const n = 12 + var wg sync.WaitGroup + errs := make([]error, n) + for i := 0; i < n; i++ { + wg.Add(1) + go func(i int) { + defer wg.Done() + _, errs[i] = manager.PutRecord(ctx, testDID, testCollection, testRKey(i), testRecord(fmt.Sprintf("post %d", i))) + }(i) + } + wg.Wait() + for i, err := range errs { + require.NoError(t, err, "concurrent put %d", i) + } + + // All records landed and revs are strictly increasing in seq order. + rows, err := database.QueryContext(ctx, + `SELECT rev FROM firehose_events WHERE did = $1 ORDER BY seq`, testDID) + require.NoError(t, err) + defer rows.Close() + var prev string + var count int + for rows.Next() { + var rev string + require.NoError(t, rows.Scan(&rev)) + assert.Greater(t, rev, prev, "revs must be strictly increasing in seq order") + prev = rev + count++ + } + require.NoError(t, rows.Err()) + assert.Equal(t, n, count) + + carBytes, err := manager.ExportCAR(ctx, testDID) + require.NoError(t, err) + _, loaded, err := indigorepo.LoadRepoFromCAR(ctx, bytes.NewReader(carBytes)) + require.NoError(t, err) + for i := 0; i < n; i++ { + nsid, rk, err := syntax.ParseRepoPath(testCollection + "/" + testRKey(i)) + require.NoError(t, err) + _, _, err = loaded.GetRecordBytes(ctx, nsid, rk) + require.NoError(t, err, "record %d must survive concurrent writes", i) + } +} + +func TestPutRecord_RevContinuityAcrossManagerRestart(t *testing.T) { + // A fresh Manager over the same database simulates a process restart: + // rev monotonicity must come from repo_state (NextRev off the stored + // rev), not from any in-memory clock the old Manager held. + manager1, database, key := testManager(t) + ctx := t.Context() + + first, err := manager1.PutRecord(ctx, testDID, testCollection, testRKey(1), testRecord("before restart")) + require.NoError(t, err) + + manager2, err := NewManager(database, &staticKeys{key: key}, nil) + require.NoError(t, err) + + second, err := manager2.PutRecord(ctx, testDID, testCollection, testRKey(2), testRecord("after restart")) + require.NoError(t, err) + assert.Greater(t, second.Rev, first.Rev, + "rev must stay monotonic across a restart (NextRev derives from the stored rev)") + + // Exactly one new firehose event, chained to the pre-restart head. + rows, err := database.QueryContext(ctx, + `SELECT rev, since_rev FROM firehose_events WHERE did = $1 ORDER BY seq`, testDID) + require.NoError(t, err) + defer rows.Close() + type ev struct { + rev string + sinceRev *string + } + var events []ev + for rows.Next() { + var e ev + require.NoError(t, rows.Scan(&e.rev, &e.sinceRev)) + events = append(events, e) + } + require.NoError(t, rows.Err()) + require.Len(t, events, 2, "one event per commit, before and after the restart") + assert.Equal(t, second.Rev, events[1].rev) + require.NotNil(t, events[1].sinceRev, "the post-restart commit is not a genesis") + assert.Equal(t, first.Rev, *events[1].sinceRev, + "post-restart event must chain to the pre-restart head rev") +} + +func TestGenesisCommit_SerializedAcrossManagers(t *testing.T) { + // Two separate Managers (simulating two processes: no shared per-DID + // mutex) race first-writes to the same new DID. The global commit + // advisory lock — not the repo_state row lock, which cannot lock a + // missing row — must serialize genesis, so exactly one genesis event + // exists and no commit gets silently overwritten. + managerA, database, key := testManager(t) + managerB, err := NewManager(database, &staticKeys{key: key}, nil) + require.NoError(t, err) + ctx := t.Context() + + const perManager = 3 + var wg sync.WaitGroup + errs := make([]error, 2*perManager) + for i := 0; i < perManager; i++ { + for j, manager := range []*Manager{managerA, managerB} { + wg.Add(1) + go func(idx int, m *Manager) { + defer wg.Done() + _, errs[idx] = m.PutRecord(ctx, testDID, testCollection, testRKey(idx), + testRecord(fmt.Sprintf("post %d", idx))) + }(i*2+j, manager) + } + } + wg.Wait() + for i, err := range errs { + require.NoError(t, err, "concurrent cross-manager put %d", i) + } + + // Exactly one genesis event (since_rev IS NULL). + var genesis int + require.NoError(t, database.QueryRowContext(ctx, + `SELECT COUNT(*) FROM firehose_events WHERE did = $1 AND since_rev IS NULL`, + testDID).Scan(&genesis)) + assert.Equal(t, 1, genesis, "exactly one genesis commit despite the cross-process race") + + // Revs strictly ordered in seq order across both managers. + rows, err := database.QueryContext(ctx, + `SELECT rev FROM firehose_events WHERE did = $1 ORDER BY seq`, testDID) + require.NoError(t, err) + defer rows.Close() + var prev string + var count int + for rows.Next() { + var rev string + require.NoError(t, rows.Scan(&rev)) + assert.Greater(t, rev, prev, "revs must be strictly increasing in seq order") + prev = rev + count++ + } + require.NoError(t, rows.Err()) + assert.Equal(t, 2*perManager, count) + + // Every record is reachable from the final head: no write was lost to a + // genesis overwrite. + carBytes, err := managerA.ExportCAR(ctx, testDID) + require.NoError(t, err) + _, loaded, err := indigorepo.LoadRepoFromCAR(ctx, bytes.NewReader(carBytes)) + require.NoError(t, err) + for i := 0; i < 2*perManager; i++ { + nsid, rk, err := syntax.ParseRepoPath(testCollection + "/" + testRKey(i)) + require.NoError(t, err) + _, _, err = loaded.GetRecordBytes(ctx, nsid, rk) + require.NoError(t, err, "record %d must be reachable from the final head", i) + } +} + +func TestFirehoseEvents_NonGenesisCARSlicesCarryRecordBlocks(t *testing.T) { + // The genesis CAR slice is covered by TestFirehoseEvents_AtomicWithCommits + // (it parses as a complete mini-repo). This pins the NON-genesis slices: + // a subscribeRepos consumer materializes updates from the event's blocks + // alone, so the update slice must physically contain the new record + // block — a commit-only slice would parse but be useless. + manager, database, _ := testManager(t) + ctx := t.Context() + rkey := testRKey(9) + + _, err := manager.PutRecord(ctx, testDID, testCollection, rkey, testRecord("v1")) + require.NoError(t, err) + update, err := manager.PutRecord(ctx, testDID, testCollection, rkey, testRecord("v2")) + require.NoError(t, err) + deleted, err := manager.DeleteRecord(ctx, testDID, testCollection, rkey) + require.NoError(t, err) + + for _, tc := range []struct { + name string + res *CommitResult + wantRecordCID string // required in the slice when non-empty + }{ + {name: "update", res: update, wantRecordCID: update.RecordCID}, + {name: "delete", res: deleted}, + } { + t.Run(tc.name, func(t *testing.T) { + var carBytes []byte + require.NoError(t, database.QueryRowContext(ctx, + `SELECT car FROM firehose_events WHERE seq = $1`, tc.res.Seq).Scan(&carBytes)) + + // Indigo must parse the slice, with the commit block as root. + commit, root, err := indigorepo.LoadCommitFromCAR(ctx, bytes.NewReader(carBytes)) + require.NoError(t, err) + assert.Equal(t, tc.res.CommitCID, root.String(), "CAR root must be the commit block") + assert.Equal(t, tc.res.Rev, commit.Rev) + + // Enumerate the physical blocks in stream order. + reader, err := car.NewCarReader(bytes.NewReader(carBytes)) + require.NoError(t, err) + var cids []string + for { + blk, err := reader.Next() + if err != nil { + break // io.EOF ends the stream + } + cids = append(cids, blk.Cid().String()) + } + require.NotEmpty(t, cids) + assert.Equal(t, tc.res.CommitCID, cids[0], + "commit (root) block must be physically first in the slice") + + if tc.wantRecordCID != "" { + assert.Contains(t, cids, tc.wantRecordCID, + "update slice must carry the new record block, not just the commit") + } + }) + } +} + +func TestRepos_IsolatedPerDID(t *testing.T) { + manager, _, _ := testManager(t) + ctx := t.Context() + + rkey := testRKey(6) + _, err := manager.PutRecord(ctx, testDID, testCollection, rkey, testRecord("mine")) + require.NoError(t, err) + _, err = manager.PutRecord(ctx, testOtherDID, testCollection, rkey, testRecord("theirs")) + require.NoError(t, err) + + mine, _, err := manager.GetRecord(ctx, testDID, testCollection, rkey) + require.NoError(t, err) + theirs, _, err := manager.GetRecord(ctx, testOtherDID, testCollection, rkey) + require.NoError(t, err) + assert.Equal(t, "mine", mine["text"]) + assert.Equal(t, "theirs", theirs["text"]) + + headA, _, err := manager.Head(ctx, testDID) + require.NoError(t, err) + headB, _, err := manager.Head(ctx, testOtherDID) + require.NoError(t, err) + assert.NotEqual(t, headA, headB) +} diff --git a/internal/repo/tid.go b/internal/repo/tid.go new file mode 100644 index 0000000..dcf16ce --- /dev/null +++ b/internal/repo/tid.go @@ -0,0 +1,82 @@ +package repo + +import ( + "crypto/sha256" + "encoding/binary" + "fmt" + "time" + + "github.com/bluesky-social/indigo/atproto/syntax" + + "tidepool/internal/errors" +) + +// TID generation for the virtual repo layer. Two distinct kinds: +// +// - Commit revs use clock TIDs that must increase monotonically per repo. +// NextRev derives them from the previous stored rev, so monotonicity +// survives process restarts and wall-clock regressions without any +// in-memory clock state. +// +// - Record keys use deterministic content TIDs (PLAN.md locked decision 4): +// the timestamp half comes from the AP object's `published` time and the +// 10 clock-ID bits from a hash of the canonical AP id. Re-ingesting the +// same AP object therefore always produces the same rkey (and at-uri), +// across process restarts. Task 05 calls DeterministicTID for every +// materialized record. + +// NextRev returns the rev TID for a repo's next commit, strictly greater +// than prevRev. An empty prevRev (genesis commit) starts a fresh clock at +// the current time. A malformed stored prevRev is a bug in our own data, so +// it surfaces as an error rather than silently restarting the clock (which +// could emit a non-monotonic rev). +func NextRev(prevRev string) (syntax.TID, error) { + if prevRev == "" { + clk := syntax.NewTIDClock(0) + return clk.Next(), nil + } + prev, err := syntax.ParseTID(prevRev) + if err != nil { + return "", err + } + clk := syntax.ClockFromTID(prev) + return clk.Next(), nil +} + +// DeterministicTID builds the content TID used as a record key for a +// materialized AP object. published supplies the timestamp bits (so records +// sort by original publication time); the clock-ID bits come from a SHA-256 +// hash of the canonical AP id, so the same object always maps to the same +// rkey across process restarts. +// +// Collision resistance: 10 clock-ID bits alone hit birthday collisions at +// roughly ~40 objects sharing a timestamp, and AP `published` is usually +// second-precision. So when published has no sub-second component, the +// microsecond field is filled deterministically from further bytes of the +// sha256(ap_id) hash (range 0..999999, ~20 extra bits of entropy), keeping +// the TID idempotent, format-valid, and sorted within the correct second. +// Objects with genuine sub-second precision keep their real microseconds. +// Caveat: for second-precision inputs, ordering WITHIN a second is hash +// order, not publication order. +// +// It fails closed on invalid times (zero or pre-Unix-epoch published → +// errors.IsValidation): callers must gate on task 02's ap.Time.OK() so a +// present-but-malformed `published` is never turned into a zero-time TID +// (which would collide and mis-sort). +func DeterministicTID(published time.Time, canonicalAPID string) (syntax.TID, error) { + if published.IsZero() { + return "", errors.NewValidationError("published", + "must not be the zero time (gate on ap.Time.OK() before deriving rkeys)") + } + if published.Unix() < 0 { + return "", errors.NewValidationError("published", + fmt.Sprintf("%s precedes the Unix epoch", published.UTC().Format(time.RFC3339))) + } + sum := sha256.Sum256([]byte(canonicalAPID)) + clockID := uint(binary.BigEndian.Uint16(sum[:2]) & 0x3FF) + micros := published.UTC().UnixMicro() + if published.Nanosecond() == 0 { + micros += int64(binary.BigEndian.Uint32(sum[2:6]) % 1_000_000) + } + return syntax.NewTID(micros, clockID), nil +} diff --git a/internal/repo/tid_test.go b/internal/repo/tid_test.go new file mode 100644 index 0000000..cd2a2ce --- /dev/null +++ b/internal/repo/tid_test.go @@ -0,0 +1,164 @@ +package repo + +import ( + "testing" + "time" + + "github.com/bluesky-social/indigo/atproto/syntax" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "tidepool/internal/errors" +) + +// mustDeterministicTID keeps the happy-path tests terse. +func mustDeterministicTID(t *testing.T, published time.Time, apID string) syntax.TID { + t.Helper() + tid, err := DeterministicTID(published, apID) + require.NoError(t, err) + return tid +} + +func TestDeterministicTID_StableAcrossCalls(t *testing.T) { + published := time.Date(2026, 5, 1, 12, 30, 45, 123456000, time.UTC) + apID := "https://lemmy.world/post/12345" + + first := mustDeterministicTID(t, published, apID) + second := mustDeterministicTID(t, published, apID) + + assert.Equal(t, first, second, + "same AP id + published must always produce the same TID (idempotent re-ingestion)") + + _, err := syntax.ParseTID(first.String()) + require.NoError(t, err, "deterministic TID must be format-valid") + assert.Equal(t, published, first.Time(), + "genuine sub-second precision must round-trip the published time exactly") +} + +func TestDeterministicTID_DifferentIDsDiffer(t *testing.T) { + published := time.Date(2026, 5, 1, 12, 30, 45, 0, time.UTC) + + a := mustDeterministicTID(t, published, "https://lemmy.world/post/1") + b := mustDeterministicTID(t, published, "https://lemmy.world/post/2") + + assert.NotEqual(t, a, b, + "objects published in the same second must still get distinct rkeys via the id-hash bits") +} + +func TestDeterministicTID_SecondPrecisionGetsHashMicros(t *testing.T) { + // AP `published` is usually second-precision; the microsecond field is + // then filled from the ap_id hash, so a same-second collision needs the + // ~20 micro bits AND the 10 clock-ID bits to collide at once. + published := time.Date(2026, 5, 1, 12, 30, 45, 0, time.UTC) + + a := mustDeterministicTID(t, published, "https://lemmy.world/post/1") + assert.Equal(t, a, mustDeterministicTID(t, published, "https://lemmy.world/post/1"), + "hash-filled micros must be deterministic") + + got := a.Time() + assert.False(t, got.Before(published), "filled micros must stay within the published second") + assert.True(t, got.Before(published.Add(time.Second)), "filled micros must stay within the published second") + assert.NotEqual(t, published, got, + "this fixture's hash micros are non-zero; equality means the fill was not applied") + + b := mustDeterministicTID(t, published, "https://lemmy.world/post/2") + assert.NotEqual(t, a.Time(), b.Time(), + "different ids land on different micros within the second (for these fixtures)") + + _, err := syntax.ParseTID(a.String()) + require.NoError(t, err, "hash-filled TID must stay format-valid") +} + +func TestDeterministicTID_GoldenValues(t *testing.T) { + // GOLDEN VALUES — these pin the persisted-rkey algorithm itself, not + // just its properties. DeterministicTID output is stored in rkeys and + // at-uris (PLAN.md locked decision 4): if any of these assertions ever + // fails, the algorithm changed and EVERY at-uri already persisted by a + // deployed bridge breaks. Do not "fix" a diff by updating the golden + // strings unless you are knowingly migrating all stored rkeys. + // + // The values were computed by running the current implementation once + // and hard-coding its output. + + // (a) Second-precision published time: AP `published` is usually + // second-precision, so the microsecond field is filled from + // sha256(ap_id) bytes (the hash-derived-micros path). + secondPrecision := time.Date(2026, 5, 1, 12, 30, 45, 0, time.UTC) + tid := mustDeterministicTID(t, secondPrecision, "https://lemmy.world/post/12345") + assert.Equal(t, "3mks4zznhkard", tid.String(), + "hash-derived-micros path changed: every persisted at-uri would break") + + // (b) Sub-second-precision published time: genuine microseconds are + // kept verbatim (the real-micros path). + subSecond := time.Date(2026, 5, 1, 12, 30, 45, 123456000, time.UTC) + tid = mustDeterministicTID(t, subSecond, "https://lemmy.world/post/12345") + assert.Equal(t, "3mks4zzqmg2rd", tid.String(), + "real-micros path changed: every persisted at-uri would break") + + // (c) Two different ap_ids in the same second: distinct, pinned TIDs + // (both hash micros and clock-ID bits derive from the ap_id). + a := mustDeterministicTID(t, secondPrecision, "https://lemmy.world/post/1") + b := mustDeterministicTID(t, secondPrecision, "https://lemmy.world/post/2") + assert.Equal(t, "3mks522bgpxc5", a.String()) + assert.Equal(t, "3mks522eqiu4d", b.String()) + assert.NotEqual(t, a, b, + "same-second objects must keep getting distinct rkeys") +} + +func TestDeterministicTID_RejectsInvalidTimes(t *testing.T) { + _, err := DeterministicTID(time.Time{}, "https://lemmy.world/post/1") + require.Error(t, err, "zero published time must fail closed") + assert.True(t, errors.IsValidation(err)) + + _, err = DeterministicTID(time.Date(1969, 12, 31, 23, 59, 59, 0, time.UTC), "https://lemmy.world/post/1") + require.Error(t, err, "pre-epoch published time must fail closed") + assert.True(t, errors.IsValidation(err)) +} + +func TestDeterministicTID_SortsByPublishedTime(t *testing.T) { + early := mustDeterministicTID(t, time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC), "https://lemmy.world/post/9") + late := mustDeterministicTID(t, time.Date(2026, 6, 1, 0, 0, 0, 0, time.UTC), "https://lemmy.world/post/1") + + assert.Less(t, early.String(), late.String(), + "TIDs must sort by published time regardless of AP id") +} + +func TestDeterministicTID_TimezoneNormalized(t *testing.T) { + utc := time.Date(2026, 5, 1, 12, 0, 0, 0, time.UTC) + offset := utc.In(time.FixedZone("CEST", 2*3600)) + + assert.Equal(t, mustDeterministicTID(t, utc, "https://x/1"), mustDeterministicTID(t, offset, "https://x/1"), + "the same instant in different zones must produce the same TID") +} + +func TestNextRev_Genesis(t *testing.T) { + rev, err := NextRev("") + require.NoError(t, err) + _, err = syntax.ParseTID(rev.String()) + require.NoError(t, err) +} + +func TestNextRev_Monotonic(t *testing.T) { + rev, err := NextRev("") + require.NoError(t, err) + for i := 0; i < 100; i++ { + next, err := NextRev(rev.String()) + require.NoError(t, err) + assert.Greater(t, next.String(), rev.String(), "revs must strictly increase") + rev = next + } +} + +func TestNextRev_MonotonicPastFutureRev(t *testing.T) { + // A stored rev from the future (clock skew) must still yield a greater + // rev, not a smaller one. + future := syntax.NewTIDFromTime(time.Now().Add(time.Hour), 7) + next, err := NextRev(future.String()) + require.NoError(t, err) + assert.Greater(t, next.String(), future.String()) +} + +func TestNextRev_RejectsMalformedStoredRev(t *testing.T) { + _, err := NextRev("not-a-tid!") + require.Error(t, err, "corrupt stored rev must fail loudly, not restart the clock") +} diff --git a/internal/store/bridged_actors.go b/internal/store/bridged_actors.go index 12e3525..1e7965b 100644 --- a/internal/store/bridged_actors.go +++ b/internal/store/bridged_actors.go @@ -112,6 +112,18 @@ func (r *postgresBridgedActors) GetByDID(ctx context.Context, did string) (*Brid return actor, nil } +func (r *postgresBridgedActors) GetByHandle(ctx context.Context, handle string) (*BridgedActor, error) { + query := `SELECT` + bridgedActorColumns + ` FROM bridged_actors WHERE handle = $1` + actor, err := scanBridgedActor(r.db.QueryRowContext(ctx, query, handle)) + if err != nil { + if stderrors.Is(err, sql.ErrNoRows) { + return nil, errors.NewNotFoundError("bridged_actor", handle) + } + return nil, fmt.Errorf("get bridged_actor by handle %q: %w", handle, err) + } + return actor, nil +} + func (r *postgresBridgedActors) SetConsentState(ctx context.Context, apActorID string, state ConsentState) error { if !state.Valid() { return errors.NewValidationError("consent_state", fmt.Sprintf("unknown state %q", state)) diff --git a/internal/store/bridged_actors_test.go b/internal/store/bridged_actors_test.go index fb781b7..ca5e3a3 100644 --- a/internal/store/bridged_actors_test.go +++ b/internal/store/bridged_actors_test.go @@ -154,6 +154,31 @@ func TestBridgedActors_Get(t *testing.T) { assert.True(t, errors.IsNotFound(err), "expected IsNotFound, got %v", err) } +func TestBridgedActors_GetByHandle(t *testing.T) { + repo := NewBridgedActors(testDB(t)) + ctx := context.Background() + + stored, err := repo.UpsertActor(ctx, testActor()) + require.NoError(t, err) + + byHandle, err := repo.GetByHandle(ctx, stored.Handle) + require.NoError(t, err) + assert.Equal(t, stored.ID, byHandle.ID) + + _, err = repo.GetByHandle(ctx, "nobody.lemmy-world.tidepool.example") + assert.True(t, errors.IsNotFound(err), "expected IsNotFound, got %v", err) + + // Actors without a handle yet (NULL) must not match anything. + unhandled := testActor() + unhandled.APActorID = "https://lemmy.world/u/bob" + unhandled.DID = testSecondDID + unhandled.Handle = "" + _, err = repo.UpsertActor(ctx, unhandled) + require.NoError(t, err) + _, err = repo.GetByHandle(ctx, "") + assert.True(t, errors.IsNotFound(err), "empty handle must not match NULL-handle rows") +} + func TestBridgedActors_ConsentStateTransitions(t *testing.T) { repo := NewBridgedActors(testDB(t)) ctx := context.Background() diff --git a/internal/store/interfaces.go b/internal/store/interfaces.go index 99537f3..e7e0641 100644 --- a/internal/store/interfaces.go +++ b/internal/store/interfaces.go @@ -76,6 +76,11 @@ type BridgedActors interface { // GetByDID returns the actor for a bridged DID. GetByDID(ctx context.Context, did string) (*BridgedActor, error) + // GetByHandle returns the actor for a bridged handle. Task 03 uses it + // for handle-collision suffixing during minting and for + // com.atproto.identity.resolveHandle. + GetByHandle(ctx context.Context, handle string) (*BridgedActor, error) + // SetConsentState transitions the actor's consent state. Deleted is // terminal: transitioning away from ConsentStateDeleted returns an // error satisfying errors.IsValidation (re-tombstoning an already diff --git a/internal/store/migrations_test.go b/internal/store/migrations_test.go index 982cd14..a92e086 100644 --- a/internal/store/migrations_test.go +++ b/internal/store/migrations_test.go @@ -11,9 +11,9 @@ import ( ) // TestMigrations_UpDownUp proves every migration applies cleanly up AND -// down. It lives in this package (not internal/db) on purpose: `go test -// ./...` runs packages in parallel, and all tests sharing the postgres -// schema must stay in one package so they run sequentially. +// down. Packages sharing the postgres schema are serialized by +// testutil.DB's advisory lock, so tearing the schema down here cannot race +// another package's tests. func TestMigrations_UpDownUp(t *testing.T) { database := testDB(t) // already migrated up by the harness ctx := context.Background() @@ -24,7 +24,8 @@ func TestMigrations_UpDownUp(t *testing.T) { err := database.QueryRowContext(ctx, ` SELECT COUNT(*) FROM information_schema.tables WHERE table_schema = 'public' - AND table_name IN ('ap_objects', 'bridged_actors', 'communities', 'inbox_events', 'service_keys') + AND table_name IN ('ap_objects', 'bridged_actors', 'communities', 'inbox_events', 'service_keys', + 'blocks', 'repo_state', 'firehose_events') `).Scan(&remaining) require.NoError(t, err) assert.Zero(t, remaining, "down migrations must drop every Tidepool table") diff --git a/internal/store/store_test.go b/internal/store/store_test.go index a9e0c47..b108d64 100644 --- a/internal/store/store_test.go +++ b/internal/store/store_test.go @@ -1,63 +1,27 @@ package store import ( - "context" "database/sql" - "os" - "sync" "testing" "time" - "github.com/stretchr/testify/require" - - "tidepool/internal/db" + "tidepool/internal/testutil" ) // The store tests run against a real postgres database (Coves convention: // real infrastructure, no mocks). They skip cleanly when // TIDEPOOL_TEST_DATABASE_URL is unset; `make test` starts the postgres-test -// container and sets it. - -var ( - testDatabaseOnce sync.Once - testDatabase *sql.DB - testDatabaseErr error -) +// container and sets it. testutil.DB holds the cross-package advisory lock +// that serializes the packages sharing this database. // testDB returns a migrated connection to the test database, truncating all -// Tidepool tables so each test starts clean. +// tables this package touches so each test starts clean. func testDB(t *testing.T) *sql.DB { t.Helper() - - databaseURL := os.Getenv("TIDEPOOL_TEST_DATABASE_URL") - if databaseURL == "" { - // In CI a missing database must fail loudly: skipping every - // postgres-backed test would let the suite go green while testing - // nothing. - if os.Getenv("CI") != "" { - t.Fatal("CI is set but TIDEPOOL_TEST_DATABASE_URL is not; " + - "the postgres-backed store tests must run in CI") - } - t.Skip("TIDEPOOL_TEST_DATABASE_URL not set; skipping postgres-backed store tests " + - "(run `make test` to start the postgres-test container and set it)") - } - - testDatabaseOnce.Do(func() { - ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) - defer cancel() - testDatabase, testDatabaseErr = db.Open(ctx, databaseURL) - if testDatabaseErr != nil { - return - } - testDatabaseErr = db.MigrateUp(ctx, testDatabase) - }) - require.NoError(t, testDatabaseErr, "connect and migrate test database") - - _, err := testDatabase.ExecContext(context.Background(), - `TRUNCATE ap_objects, bridged_actors, communities, inbox_events, service_keys RESTART IDENTITY`) - require.NoError(t, err, "truncate test tables") - - return testDatabase + database := testutil.DB(t) + testutil.Truncate(t, database, + "ap_objects", "bridged_actors", "communities", "inbox_events", "service_keys") + return database } // Shared fixtures for readable tests. diff --git a/internal/testutil/db.go b/internal/testutil/db.go new file mode 100644 index 0000000..19e7a4c --- /dev/null +++ b/internal/testutil/db.go @@ -0,0 +1,96 @@ +// Package testutil provides the shared postgres harness for Tidepool's +// integration tests. Multiple packages (store, repo, identity, ...) test +// against the same database, and `go test ./...` runs package binaries in +// parallel — so the harness takes a postgres advisory lock for the lifetime +// of each test process, serializing the packages that share the schema. +// (Within one package, tests already run sequentially.) +package testutil + +import ( + "context" + "database/sql" + "fmt" + "os" + "strings" + "sync" + "testing" + "time" + + "tidepool/internal/db" +) + +// advisoryLockKey is arbitrary but must be shared by every package using +// the test database. It MUST stay distinct from internal/repo's +// commitAdvisoryLockKey (0x7469646570636d): this lock is session-scoped and +// held for an entire test binary, while the repo one is transaction-scoped +// and taken by every commit — sharing a key would deadlock every repo +// commit made from tests. +const advisoryLockKey int64 = 0x7469646570 // "tidep" + +var ( + once sync.Once + conn *sql.DB + // lockConn pins one session that holds the advisory lock until this + // test process exits (postgres releases advisory locks on session end). + lockConn *sql.Conn + setupErr error +) + +// DB returns a migrated connection to the test database, guarded by the +// cross-package advisory lock. It skips the test when +// TIDEPOOL_TEST_DATABASE_URL is unset — except in CI, where a missing +// database fails loudly (skipping every postgres-backed test would let the +// suite go green while testing nothing). `make test` starts the +// postgres-test container and sets the variable. +func DB(t *testing.T) *sql.DB { + t.Helper() + + databaseURL := os.Getenv("TIDEPOOL_TEST_DATABASE_URL") + if databaseURL == "" { + if os.Getenv("CI") != "" { + t.Fatal("CI is set but TIDEPOOL_TEST_DATABASE_URL is not; " + + "the postgres-backed tests must run in CI") + } + t.Skip("TIDEPOOL_TEST_DATABASE_URL not set; skipping postgres-backed tests " + + "(run `make test` to start the postgres-test container and set it)") + } + + once.Do(func() { + // Generous timeout: acquiring the advisory lock may wait for + // another package's whole test binary to finish. + ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute) + defer cancel() + + conn, setupErr = db.Open(ctx, databaseURL) + if setupErr != nil { + return + } + lockConn, setupErr = conn.Conn(ctx) + if setupErr != nil { + return + } + if _, setupErr = lockConn.ExecContext(ctx, + `SELECT pg_advisory_lock($1)`, advisoryLockKey); setupErr != nil { + return + } + setupErr = db.MigrateUp(ctx, conn) + }) + if setupErr != nil { + t.Fatalf("connect and migrate test database: %v", setupErr) + } + return conn +} + +// Truncate empties the given tables and resets their sequences, so a test +// starts from a clean slate. +func Truncate(t *testing.T, conn *sql.DB, tables ...string) { + t.Helper() + if len(tables) == 0 { + return + } + _, err := conn.ExecContext(context.Background(), + fmt.Sprintf(`TRUNCATE %s RESTART IDENTITY`, strings.Join(tables, ", "))) + if err != nil { + t.Fatalf("truncate test tables: %v", err) + } +}