diff --git a/docs/specs/README.md b/docs/specs/README.md index 52e62f4..88e8107 100644 --- a/docs/specs/README.md +++ b/docs/specs/README.md @@ -18,7 +18,7 @@ Subsystem specifications live in this directory. 2. [XRPC HTTP Surface](./xrpc.md) 3. [SQLite Storage](./storage-sqlite.md) 4. [Accounts and Auth](accounts-auth.md) -5. [Identity and Handles](identity-handles.md) +5. [Identity and Handles](./identity-handles.md) 6. [Repository Core](./repo-core.md) 7. [Record APIs](./record-apis.md) 8. [Lexicon Schema Loading and Generation](./lexicon-schemas.md) @@ -32,7 +32,7 @@ Subsystem specifications live in this directory. 16. [PDS Compatibility Against Reference Surface](./pds-compatibility.md) 17. [Public Stats Dashboard](./public-stats-dashboard.md) 18. [Documentation Viewer](./doc-viewer.md) -19. [Personal Account Backups](./personal-backups.md) +19. [Account Management Control Panel](./account-management.md) ## Source Baseline diff --git a/docs/specs/account-management.md b/docs/specs/account-management.md new file mode 100644 index 0000000..8b8774f --- /dev/null +++ b/docs/specs/account-management.md @@ -0,0 +1,443 @@ +--- +title: Account Management Control Panel +updated: 2026-06-14 +status: planned +--- + +Tempest should expose account management as a Control Panel with two separate +views: + +- user account management for the signed-in hosted account; +- admin controls for the operator who manages the Tempest instance. + +These views share visual language and some underlying context modules, but they +must not share authority. A normal account token must never authorize admin +actions. An admin token must never silently act as a hosted account. + +## Source Baseline + +Research checked on 2026-06-13: + +- AT Protocol Repository spec: +- AT Protocol Sync spec: +- AT Protocol Blob Lifecycle guide: +- AT Protocol Account Migration guide: + +- `com.atproto.sync.listBlobs` Lexicon: + +- `com.atproto.sync.getBlob` Lexicon: + +- `app.bsky.actor.getPreferences` Lexicon: + +- Official Bluesky PDS distribution: + +- Reference PDS implementation: + +- Cocoon PDS: + +- Tranquil PDS local reference: + + +## Goals + +- Provide a browser-friendly account management UI instead of requiring manual + bearer headers for every page. +- Keep user account management and admin controls as separate authenticated + surfaces. +- Let a hosted user inspect their account, repo, blobs, sessions, app passwords, + OAuth grants, security events, migration state, sequencer events, and firehose + state. +- Let an admin inspect service health, storage, hosted accounts, compatibility, + repo operations, service backups, and external account backups. +- Add personal account backups as an admin-only account-management feature for + external AT Protocol accounts controlled by the operator. +- Use LiveView for Control Panel workflows and shared context modules for + behavior. +- Use XRPC as the protocol boundary for clients and external PDS calls, not as + the primary internal API between LiveViews and Tempest contexts. +- Resolve PDS locations from DIDs and service metadata instead of hardcoding + PDS URLs in account-management config. + +## Non-goals + +- No hosted-provider moderation console in this milestone. +- No admin impersonation of a user account. +- No automatic account migration from external backup snapshots. +- No PLC updates from the personal-backup flow. +- No writes to source PDS instances during personal backup. +- No backups of DMs, notifications, AppView-only timelines, label-service state, + moderation decisions stored outside the PDS, or feed-generator state. +- No whole-PDS disaster recovery inside personal account backups. Service backup + and restore remain separate admin operations. + +## Route Model + +User account management should live under `/account/*`: + +```text +/account/login +/account/logout +/account +/account/repo +/account/blobs +/account/access +/account/security +/account/migration +/account/sequencer +/account/firehose +``` + +Admin controls should live under `/admin/*`: + +```text +/admin/login +/admin/logout +/admin +/admin/accounts +/admin/accounts/:did +/admin/storage +/admin/repo +/admin/sequencer +/admin/security +/admin/backups +/admin/backups/service +/admin/backups/accounts +/admin/backups/accounts/:id +/admin/compatibility +``` + +The implementation should replace the current controller-backed account/admin +tooling in place. Do not preserve old `/account/*` or `/admin/*` controller +routes as redirects. + +## Login And Auth + +### User Login + +The user login flow should authenticate a hosted account through the existing +account credential model: + +```text +POST /xrpc/com.atproto.server.createSession +``` + +The browser form should accept identifier, password, and any required +second-factor token for locally hosted accounts. On success, Tempest should +store only a session family or server-side session reference in the browser +session. It must not store raw access tokens or refresh tokens in the browser +session, and it must not render access tokens, refresh tokens, app-password +values, OAuth token hashes, backup-code hashes, or recovery secrets. + +The user session authorizes only `/account/*` pages and user-scoped operations. +It does not authorize `/admin/*`. + +The existing bearer-token behavior can remain for development and smoke tests, +but browser users should not need to manually attach `Authorization` headers. + +### Admin Login + +The admin identity should be configured by DID, not by a hardcoded PDS URL: + +```text +TEMPEST_ADMIN_DID=did:plc:... +``` + +`/admin/login` should resolve `TEMPEST_ADMIN_DID`, discover the account's current +PDS/auth metadata, and choose the correct authentication method. + +If the admin DID is hosted by this Tempest instance, the login can use the local +account credential/session flow and then verify that the authenticated account +DID equals `TEMPEST_ADMIN_DID`. + +If the admin DID is hosted elsewhere, including a Bluesky PDS, the login should +use AT Protocol OAuth for that DID. The browser session should store only a +server-side admin auth reference, not OAuth access tokens, refresh tokens, DPoP +keys, or raw authorization artifacts. + +`TEMPEST_ADMIN_TOKEN_HASH` may remain as a bootstrap or automation credential +for existing JSON/status checks, but it should not be the primary browser +Control Panel login model. + +Admin sessions authorize only `/admin/*` pages and admin operations. They do not +act as a hosted account and must not be accepted by account-only XRPC methods. + +### Shared Safeguards + +- Put user and admin LiveViews in separate authenticated `live_session` groups. +- Keep separate plugs for account-session auth and admin-session auth. +- Apply CSRF protection to browser forms. +- Rate-limit login attempts and manual backup triggers. +- Redact auth headers, credentials, and tokens from logs and templates. +- Require explicit confirmations for destructive admin actions. +- Store only opaque session family IDs or server-side auth references in browser + sessions. + +## LiveView And XRPC Boundary + +Control Panel pages should be LiveView-first: + +```text +LiveView + -> Tempest context module + -> storage, repo, security, backup, or external-PDS client +``` + +LiveViews should not call Tempest's own XRPC HTTP endpoints for internal admin +work. They should call context modules directly so auth, validation, error +handling, and tests stay simple. + +Use XRPC for: + +- protocol-compatible endpoints used by external clients; +- links that intentionally download protocol resources, such as repo CARs or + blobs; +- outbound calls to external PDS instances during personal backup. + +Use `Req` for outbound HTTP. + +## User Account Management View + +The user view is scoped to the signed-in hosted account. + +It should include: + +- identity summary: DID, handle, email, active state, and hosting status; +- repository summary: collections, recent records, latest commit, and CAR link; +- blob browser: temp/public blob state, download links, and header summary; +- access inventory: sessions, OAuth grants, app passwords, delegated access; +- security inventory: email state, password state, MFA, backup codes, trusted + devices, and security events; +- migration state: activation and migration-readiness status; +- sequencer view: recent events for the account; +- firehose view: recent decoded `subscribeRepos` frames and WebSocket URL. + +The first version can stay mostly read-only. Mutating user actions should be +added only where there is already a safe context API and test coverage. + +## Admin Control View + +The admin view is scoped to the Tempest instance. + +It should include: + +- service status, version, public URL, configured host, and admin auth state; +- hosted account list and per-account detail; +- storage status for SQLite, repos, blobs, backups, and object storage; +- repo operations: verify, export, and import; +- sequencer status and recent events; +- security overview, including admin-token configuration and redacted security + event summaries; +- service backup create and restore dry-run; +- compatibility matrix; +- external account backups. + +Admin pages may show aggregate hosted-account information and operational +warnings. They must avoid exposing raw credentials or token material. + +## External Account Backups + +External account backups are part of admin account management because they may +contain private preferences and deleted public data. They must not be exposed in +the user account view except as a link or status note when appropriate. + +This feature backs up other AT Protocol accounts controlled by the operator +without becoming the active PDS for those accounts. + +### Account Registry + +Add a registry for external accounts: + +```text +id +label +did +handle +source_pds_url +credential_state +last_checked_at +last_success_at +last_snapshot_id +status +status_reason +inserted_at +updated_at +``` + +`did` is the stable account identifier. `handle` is display and discovery +metadata. A backup must verify that the resolved handle still points to the DID, +but a handle change must not orphan existing snapshots. + +`source_pds_url` is the PDS used for backup reads. It must be resolved from the +DID document by default, not hardcoded. The operator may pin it for an account as +an explicit override. If discovery and the pinned source disagree, the backup +should fail closed unless the operator confirms a source update. + +### Credentials + +Support three credential states: + +```text +none +app_password +access_token +``` + +Public repo and blob backup should work with no credential. Private preference +backup requires auth. + +Credential rules: + +- Store secrets encrypted or through the existing secret-storage approach chosen + for deployment. +- Never display a stored secret after save. +- Allow credential replacement and deletion. +- Record credential kind and last verification time. +- Treat failed auth as a backup warning when public backup succeeds, not as a + failed public backup. + +### Snapshot Model + +Each backup run creates a snapshot. A snapshot is immutable after completion. + +Suggested manifest fields: + +```json +{ + "version": 1, + "account": { "did": "did:plc:...", "handle": "example.com", "sourcePds": "https://bsky.social" }, + "repo": { "carPath": "repo.car", "commit": "bafy...", "rev": "3l...", "byteSize": 12345, "sha256": "..." }, + "blobs": { "count": 10, "complete": true, "missing": [] }, + "preferences": { "included": true, "path": "preferences.json" }, + "verification": { "status": "ok", "checkedAt": "2026-06-13T00:00:00Z" } +} +``` + +Store snapshots under the existing backup storage profile. Local and S3/R2 +storage should share the same logical layout: + +```text +personal-backups/ + / + snapshots/ + -/ + manifest.json + repo.car + blobs/ + + preferences.json + verification.json +``` + +### Backup Flow + +1. Resolve account identity. +2. Determine the source PDS from the DID document or pinned account config. +3. Fetch `com.atproto.sync.getRepo?did=` from the source PDS. +4. Parse and verify the CAR. +5. Extract current commit CID, DID, rev, records, and blob references. +6. Call `com.atproto.sync.listBlobs` with pagination. +7. Fetch each blob with `com.atproto.sync.getBlob`. +8. Verify blob bytes match the expected CID. +9. If credentials are configured, call `app.bsky.actor.getPreferences`. +10. Write the snapshot to a temporary location. +11. Write manifest and verification report. +12. Atomically mark the snapshot complete. + +If blob enumeration fails but repo backup succeeds, keep the snapshot incomplete +and record missing blob state. Do not mark the snapshot complete until every +referenced available blob is either stored or explicitly recorded as missing. + +### Verification + +A completed snapshot must prove: + +- the CAR root points at a commit object; +- the commit DID matches the registered account DID; +- the commit signature verifies against the resolved DID document; +- the repo MST is complete for the exported commit; +- record paths and CIDs pass existing repo-core validation; +- blob CIDs discovered from records and `listBlobs` have matching stored bytes; +- preference JSON was fetched with auth when credentials were enabled; +- the manifest hashes match files on disk or in object storage. + +Verification should be callable without contacting the source PDS, except for an +optional identity freshness check. Offline verification is the point of a backup. + +### Scheduling + +Manual backup comes first. Scheduled backup can follow after the manual flow is +stable. + +Scheduling rules: + +- Run one account backup at a time by default. +- Use `Task.async_stream/3` with bounded concurrency only for blob downloads. +- Persist backup run state so an interrupted run can be marked failed or resumed. +- Do not retry auth failures without operator action. +- Use exponential backoff for transient source PDS or object-storage errors. + +### Storage And Retention + +Retention should be explicit per account: + +```text +keep_all +keep_last_n +keep_for_days +``` + +Default to `keep_last_n=3` for scheduled backups. Manual snapshots may be pinned +to prevent deletion. + +Storage reporting should include: + +- repo CAR bytes; +- blob bytes; +- preference bytes; +- manifest and verification bytes; +- total per account; +- total across personal backups. + +The snapshot manifest should stay useful if Tempest later changes database or +object-storage internals. Do not make an internal row ID or storage backend the +only way to understand a backup. + +## Security + +Backups may contain private preference data and deleted public data that still +exists in an older snapshot. Treat bundles as sensitive. + +Required safeguards: + +- account auth for all user account-management routes; +- admin auth for all admin routes and personal-backup routes; +- no public snapshot listing; +- no public bundle downloads; +- no credential values in logs or templates; +- redacted error messages for auth headers and tokens; +- rate limits on login and manual backup triggers; +- clear warning before deleting snapshots. + +## HTTP Verification + +Future smoke tests: + +```bash +hurl --test --jobs 1 \ + --variable base_url=http://localhost:4000 \ + --variable suffix="$(date +%s)" \ + test/smoke/account-management.hurl + +hurl --test --jobs 1 \ + --variable base_url=http://localhost:4000 \ + --variable admin_did="$TEMPEST_ADMIN_DID" \ + --variable admin_login_mode=fixture_oauth \ + test/smoke/account-management-admin.hurl +``` + +The user smoke test should cover account login, user account dashboard access, +and rejection from admin routes. The admin smoke test should cover admin login, +hosted-account inspection, external account registration, public repo snapshot +creation, blob backup, credentialed preferences backup against a fixture server, +snapshot verification, export bundle creation, and deletion of an unpinned +snapshot. diff --git a/docs/specs/personal-backups.md b/docs/specs/personal-backups.md deleted file mode 100644 index 709a063..0000000 --- a/docs/specs/personal-backups.md +++ /dev/null @@ -1,288 +0,0 @@ ---- -title: Personal Account Backups -updated: 2026-06-13 -status: planned ---- - -Tempest should be able to back up other AT Protocol accounts controlled by the -operator without becoming the active PDS for those accounts. - -This feature is custody and archive work. It must not update identity, submit PLC -operations, activate accounts, write records to source PDS instances, or imply -that Tempest is hosting the backed-up account. - -## Source Baseline - -Research checked on 2026-06-13: - -- AT Protocol Repository spec: -- AT Protocol Sync spec: -- AT Protocol Blob Lifecycle guide: -- AT Protocol Account Migration guide: - -- `com.atproto.sync.listBlobs` Lexicon: - -- `com.atproto.sync.getBlob` Lexicon: - -- `app.bsky.actor.getPreferences` Lexicon: - -- Official Bluesky PDS distribution: - -- Reference PDS implementation: - -- Cocoon PDS: - -- Tranquil PDS local reference: - - -The repository spec defines full repo exports as CAR files suitable for sync, -offline backup, and migration. The sync spec exposes unauthenticated -`com.atproto.sync.getRepo` for public repository export. Blob backup should use -`com.atproto.sync.listBlobs` and `com.atproto.sync.getBlob`; both are PDS -endpoints and do not require auth for public blobs. Private preferences require -auth through `app.bsky.actor.getPreferences`. - -The Tranquil local reference reinforces two design constraints. First, operator -guidance there treats repo CAR export, blob download, preference export, and -separately held rotation keys as distinct backup concerns. Second, Tranquil's -history includes an `account_backups` table with `storage_key`, repo root CID, -rev, block count, size, and created time, followed later by a migration that -dropped the table. Treat that as a warning to keep Tempest's first account -backup format portable and manifest-driven instead of tightly coupling it to -one internal storage engine. - -## Goals - -- Register external accounts by DID and handle. -- Back up public repository state as immutable CAR snapshots. -- Back up public blobs associated with the account. -- Back up private preferences when the operator supplies account credentials. -- Verify each snapshot before marking it complete. -- Export a portable bundle containing CAR, blobs, preferences, manifest, and - verification report. -- Keep this feature separate from account migration and active hosting. - -## Non-goals - -- No PLC updates. -- No identity migration. -- No automatic activation of backed-up accounts on Tempest. -- No writes to the source PDS in the first version. -- No backups of DMs, notifications, AppView-only timelines, label-service state, - moderation decisions stored outside the PDS, or feed-generator state. -- No broad network crawler. The operator must explicitly add each account. -- No whole-PDS disaster recovery in this feature. Tempest's service backup and - restore work remains separate from per-account personal snapshots. - -## Account Registry - -Add a registry for external accounts. Each entry should include: - -```text -id -label -did -handle -source_pds_url -credential_state -last_checked_at -last_success_at -last_snapshot_id -status -status_reason -inserted_at -updated_at -``` - -`did` is the stable account identifier. `handle` is display and discovery -metadata. A backup must verify that the resolved handle still points to the DID, -but a handle change must not orphan existing snapshots. - -`source_pds_url` is the PDS used for backup reads. It can be discovered from the -DID document, but the operator may pin it for an account. If discovery and the -pinned source disagree, the backup should fail closed unless the operator -confirms a source update. - -## Credentials - -Support three credential states: - -```text -none -app_password -access_token -``` - -Public repo and blob backup should work with no credential. Private preference -backup requires auth. - -Credential rules: - -- Store secrets encrypted or through the existing secret-storage approach chosen - for deployment. -- Never display a stored secret after save. -- Allow credential replacement and deletion. -- Record credential kind and last verification time. -- Treat failed auth as a backup warning when public backup succeeds, not as a - failed public backup. - -## Snapshot Model - -Each backup run creates a snapshot. A snapshot is immutable after completion. - -Suggested manifest fields: - -```json -{ - "version": 1, - "account": { "did": "did:plc:...", "handle": "example.com", "sourcePds": "https://bsky.social" }, - "repo": { "carPath": "repo.car", "commit": "bafy...", "rev": "3l...", "byteSize": 12345, "sha256": "..." }, - "blobs": { "count": 10, "complete": true, "missing": [] }, - "preferences": { "included": true, "path": "preferences.json" }, - "verification": { "status": "ok", "checkedAt": "2026-06-13T00:00:00Z" } -} -``` - -Store snapshots under the existing backup storage profile. Local and S3/R2 -storage should share the same logical layout: - -```text -personal-backups/ - / - snapshots/ - -/ - manifest.json - repo.car - blobs/ - - preferences.json - verification.json -``` - -## Backup Flow - -1. Resolve account identity. -2. Determine the source PDS from the DID document or pinned account config. -3. Fetch `com.atproto.sync.getRepo?did=` from the source PDS. -4. Parse and verify the CAR. -5. Extract current commit CID, DID, rev, records, and blob references. -6. Call `com.atproto.sync.listBlobs` with pagination. -7. Fetch each blob with `com.atproto.sync.getBlob`. -8. Verify blob bytes match the expected CID. -9. If credentials are configured, call `app.bsky.actor.getPreferences`. -10. Write the snapshot to a temporary location. -11. Write manifest and verification report. -12. Atomically mark the snapshot complete. - -If blob enumeration fails but repo backup succeeds, keep the snapshot incomplete -and record missing blob state. Do not mark the snapshot complete until every -referenced available blob is either stored or explicitly recorded as missing. - -## Verification - -A completed snapshot must prove: - -- the CAR root points at a commit object; -- the commit DID matches the registered account DID; -- the commit signature verifies against the resolved DID document; -- the repo MST is complete for the exported commit; -- record paths and CIDs pass existing repo-core validation; -- blob CIDs discovered from records and `listBlobs` have matching stored bytes; -- preference JSON was fetched with auth when credentials were enabled; -- the manifest hashes match files on disk or in object storage. - -Verification should be callable without contacting the source PDS, except for an -optional identity freshness check. Offline verification is the point of a backup. - -## Admin UI - -Add an admin-only account backup area: - -```text -/admin/backups/accounts -/admin/backups/accounts/:id -``` - -The UI should show: - -- registered accounts; -- credential state, without secret values; -- latest snapshot status; -- backup now action; -- verification action; -- snapshot list; -- missing blob report; -- export bundle download or object-storage location; -- source PDS and identity mismatch warnings. - -The operator account UI may link to this area, but external account backups are -admin-only in the first version. - -## Scheduling - -Manual backup comes first. Scheduled backup can follow after the manual flow is -stable. - -Scheduling rules: - -- Run one account backup at a time by default. -- Use `Task.async_stream/3` with bounded concurrency only for blob downloads. -- Persist backup run state so an interrupted run can be marked failed or resumed. -- Do not retry auth failures without operator action. -- Use exponential backoff for transient source PDS or object-storage errors. - -## Storage and Retention - -Retention should be explicit per account: - -```text -keep_all -keep_last_n -keep_for_days -``` - -Default to `keep_last_n=3` for scheduled backups. Manual snapshots may be pinned -to prevent deletion. - -Storage reporting should include: - -- repo CAR bytes; -- blob bytes; -- preference bytes; -- manifest and verification bytes; -- total per account; -- total across personal backups. - -The snapshot manifest should stay useful if Tempest later changes database or -object-storage internals. Do not make an internal row ID or storage backend the -only way to understand a backup. - -## Security - -Backups may contain private preference data and deleted public data that still -exists in an older snapshot. Treat bundles as sensitive. - -Required safeguards: - -- admin auth for all personal backup routes and APIs; -- no public snapshot listing; -- no public bundle downloads; -- no credential values in logs or templates; -- redacted error messages for auth headers and tokens; -- rate limits on manual backup triggers; -- clear warning before deleting snapshots. - -## HTTP Verification - -Future smoke test: - -```bash -hurl --test --jobs 1 \ - --variable base_url=http://localhost:4000 \ - --variable admin_token="$ADMIN_TOKEN" \ - test/smoke/personal-backups.hurl -``` - -The smoke test should cover account registration, public repo snapshot creation, -blob backup, credentialed preferences backup against a fixture server, snapshot -verification, export bundle creation, and deletion of an unpinned snapshot. diff --git a/docs/tasks/18-account-management.md b/docs/tasks/18-account-management.md new file mode 100644 index 0000000..4864576 --- /dev/null +++ b/docs/tasks/18-account-management.md @@ -0,0 +1,212 @@ +--- +title: Milestone 18 - Account Management Control Panel +specs: + - ../specs/account-management.md + - ../specs/admin-operations.md + - ../specs/storage-sqlite.md + - ../specs/security-oauth.md +references: + - ../reference/account-migration.md + - ../reference/blobs.md + - ../reference/repo-core.md + - ../reference/admin-operations.md +--- + +Goal: turn the current account and admin development tools into a browser-usable +Control Panel with separate user account management and admin control views. +External personal backups become an admin-only account-management feature. + +## Auth And Routing + +- [ ] T18-01: Add a browser-friendly account login page at `/account/login` + backed by the existing account credential/session flow. +- [ ] T18-02: Add account logout and an account browser-session plug that + authorizes `/account/*` from a session family or server-side session + reference without requiring manual bearer headers. +- [ ] T18-03: Preserve bearer-token access for existing account tool smoke tests + while ensuring browser sessions never store or render access or refresh + tokens. +- [ ] T18-04: Add `TEMPEST_ADMIN_DID` config and validation. Admin browser auth + must be anchored to this DID rather than a hardcoded PDS URL. +- [ ] T18-05: Add an admin login page at `/admin/login` that resolves + `TEMPEST_ADMIN_DID`, discovers the current auth method, and authenticates + either through local account login or AT Protocol OAuth. +- [ ] T18-06: Store only a server-side admin auth reference in the browser + session. Do not store raw admin tokens, OAuth access tokens, refresh + tokens, DPoP keys, or authorization artifacts in the browser session. +- [ ] T18-07: Keep `TEMPEST_ADMIN_TOKEN_HASH` available only as a bootstrap or + automation credential where still needed by JSON/status checks. +- [ ] T18-08: Add admin logout and an admin browser-session plug that accepts a + valid admin session and, for automation-only paths, the configured admin + bearer token. +- [ ] T18-09: Put user and admin LiveViews in separate authenticated + `live_session` groups. +- [ ] T18-10: Replace the current controller-backed account/admin tooling in + place. Do not preserve old `/account/*` or `/admin/*` controller routes as + redirects. +- [ ] T18-11: Add tests proving account sessions cannot access `/admin/*` and + admin sessions cannot act as account auth for account-only XRPC methods. + +## User Account Management + +- [ ] T18-12: Convert the existing `/account` dashboard into a LiveView Control + Panel page with identity, repository, blob, access, security, migration, + sequencer, and firehose navigation. +- [ ] T18-13: Convert `/account/repo` to LiveView using existing repo-storage + context helpers for collections, recent records, latest commit, and CAR + download links. +- [ ] T18-14: Convert `/account/blobs` to LiveView using existing blob context + helpers for temp/public blob state and public download links. +- [ ] T18-15: Convert `/account/access` and `/account/security` to LiveView + inventory pages that never render token, app-password, OAuth, backup-code, + or recovery secrets. +- [ ] T18-16: Convert `/account/migration`, `/account/sequencer`, and + `/account/firehose` to LiveView pages with scoped account data. +- [ ] T18-17: Add account-management ConnCase or LiveView tests for login, + logout, route auth, key element IDs, and redacted secret output. + +## Admin Control Panel + +- [ ] T18-18: Convert the existing `/admin` dashboard into a LiveView Control + Panel page for service status, hosted accounts, sequencer status, storage, + and compatibility warnings. +- [ ] T18-19: Add `/admin/accounts` and `/admin/accounts/:did` for hosted + account inspection, using admin auth only. +- [ ] T18-20: Convert `/admin/storage`, `/admin/repo`, `/admin/backups`, and + `/admin/compatibility` to LiveView or keep thin controller actions where + file downloads/forms make that simpler. +- [ ] T18-21: Keep admin operations backed by context modules rather than + calling Tempest's own XRPC HTTP endpoints internally. +- [ ] T18-22: Add confirmations and CSRF-protected forms for admin mutations + such as repo import, backup create, restore dry-run, prune, and delete. +- [ ] T18-23: Add admin tests for local-admin login, external-admin OAuth login + with a fixture auth server, bearer-token automation access, account-token + rejection, route rendering, and mutation confirmation flows. + +## External Account Backups + +- [ ] T18-24: Add a `Tempest.PersonalBackups` context and migrations for + external backup accounts, backup runs, immutable snapshots, blob records, + credentials, and retention settings. +- [ ] T18-25: Add external account registration with DID, handle, optional + pinned source PDS URL, label, credential state, and status fields. The + default source PDS must be resolved from the DID document, not hardcoded. +- [ ] T18-26: Add identity/source verification that resolves handle and DID + document, verifies `#atproto_pds`, and fails closed on mismatched pinned + source PDS values. +- [ ] T18-27: Add credential storage for no-auth, app-password, and access-token + modes. Store secrets defensively, never render them, and allow rotation and + deletion. +- [ ] T18-28: Add a source PDS client using `Req` for + `com.atproto.sync.getRepo`, `com.atproto.sync.listBlobs`, + `com.atproto.sync.getBlob`, and `app.bsky.actor.getPreferences`. +- [ ] T18-29: Add CAR snapshot creation that stores `repo.car`, commit CID, rev, + byte size, hash, source PDS, handle, and DID. +- [ ] T18-30: Reuse repo-core verification to validate commit DID, commit + signature, MST completeness, record paths, record CIDs, and CAR integrity. +- [ ] T18-31: Extract blob references from repo records and merge them with + paginated `listBlobs` output. +- [ ] T18-32: Add bounded concurrent blob download with CID verification, + missing-blob recording, and retry handling for transient source failures. +- [ ] T18-33: Add credentialed private preference backup through + `app.bsky.actor.getPreferences`, with auth failures reported separately + from public repo/blob backup status. +- [ ] T18-34: Write immutable snapshot manifests and verification reports, first + to a temporary workspace and then atomically mark snapshots complete. +- [ ] T18-35: Store personal backup snapshots through the existing local and + S3/R2 backup storage shape. +- [ ] T18-36: Add retention policies: keep all, keep last N, keep for days, and + pinned snapshots. +- [ ] T18-37: Add portable export bundle creation containing manifest, repo CAR, + blobs, preferences JSON when present, and verification report. +- [ ] T18-38: Add offline snapshot verification that can run without contacting + the source PDS. +- [ ] T18-39: Add tests proving a snapshot can be understood from its manifest + and files without relying on Tempest database rows. +- [ ] T18-40: Add Mix tasks for backup, verify, list snapshots, export bundle, + prune, and show account backup status. +- [ ] T18-41: Add admin-only LiveView routes for external backup account list, + detail, create, edit, delete, backup now, verify, prune, and export. +- [ ] T18-42: Add admin UI for credential state, latest backup status, missing + blobs, snapshot history, storage totals, and source identity warnings. +- [ ] T18-43: Add manual backup locking so two runs cannot mutate the same + account snapshot workspace at the same time. +- [ ] T18-44: Add optional scheduled backups after manual backups are stable, + with one-account-at-a-time default scheduling and persisted run state. + +## Tests And Docs + +- [ ] T18-45: Add unit tests for account registration, credential redaction, + manifest writing, retention pruning, and snapshot state transitions. +- [ ] T18-46: Add fixture-server integration tests for `getRepo`, `listBlobs`, + `getBlob`, preferences auth success, preferences auth failure, missing + blobs, bad CIDs, bad CARs, and source identity mismatch. +- [ ] T18-47: Add admin LiveView tests for route auth, create/edit forms, + backup-now action, verification action, export action, and deletion + confirmation. +- [ ] T18-48: Add Hurl smoke test `test/smoke/account-management.hurl` for user + login and user Control Panel access. +- [ ] T18-49: Add Hurl smoke test `test/smoke/account-management-admin.hurl` for + admin login, admin route auth, and external backup flows. +- [ ] T18-50: Add reference documentation after implementation describing the + account-management routes, login model, backup format, export limits, + security model, and operational checks. + +## Integration Tests + +- User login creates an account browser session containing only a session family + or server-side session reference. +- User logout clears account browser-session access. +- Admin login resolves `TEMPEST_ADMIN_DID` and succeeds through local account + auth or AT Protocol OAuth, depending on where the DID is hosted. +- Admin login creates an admin browser session containing only a server-side + admin auth reference. +- Admin logout clears admin browser-session access. +- Account sessions cannot access admin pages. +- Admin sessions cannot act as account XRPC auth. +- Account and admin browser sessions never store raw access tokens, refresh + tokens, OAuth artifacts, DPoP keys, or raw admin tokens. +- PDS URLs are resolved from DIDs and service metadata unless an operator + explicitly pins a source PDS for an external backup account. +- Public repo backup succeeds without credentials. +- Blob backup stores every available listed or referenced blob. +- CID mismatch marks a snapshot failed. +- Missing blobs are recorded and visible to the operator. +- Private preferences are included only when valid credentials are configured. +- Auth failures do not leak secrets and do not destroy public backup output. +- Offline verification catches corrupted CAR, blob, manifest, and preferences + files. +- Retention pruning never deletes pinned snapshots. +- Admin routes reject unauthenticated, account-token, and normal app-password + requests. + +## HTTP Verification + +```bash +hurl --test --jobs 1 \ + --variable base_url=http://localhost:4000 \ + --variable suffix="$(date +%s)" \ + test/smoke/account-management.hurl + +hurl --test --jobs 1 \ + --variable base_url=http://localhost:4000 \ + --variable admin_did="$TEMPEST_ADMIN_DID" \ + --variable admin_login_mode=fixture_oauth \ + test/smoke/account-management-admin.hurl +``` + +## Implementation Notes + +Keep LiveViews thin. Account and admin pages should call Tempest context modules +directly rather than calling Tempest's own XRPC HTTP routes internally. + +Use XRPC for protocol-compatible client endpoints and outbound reads from source +PDS instances. Use `Req` for outbound HTTP. + +Keep external account backups read-only against source PDS instances in the +first version. The backup client may authenticate to read private preferences, +but it must not write records, submit PLC operations, activate accounts, +deactivate accounts, or call migration endpoints. + +Prefer portable snapshot bundles over direct restore. Direct restore into +Tempest belongs in migration work after backup verification has real use. diff --git a/docs/tasks/18-personal-backups.md b/docs/tasks/18-personal-backups.md deleted file mode 100644 index 896f9c1..0000000 --- a/docs/tasks/18-personal-backups.md +++ /dev/null @@ -1,110 +0,0 @@ ---- -title: Milestone 18 - Personal Account Backups -specs: - - ../specs/personal-backups.md - - ../specs/admin-operations.md - - ../specs/storage-sqlite.md - - ../specs/security-oauth.md -references: - - ../reference/account-migration.md - - ../reference/blobs.md - - ../reference/repo-core.md - - ../reference/admin-operations.md ---- - -Goal: let the operator back up other AT Protocol accounts they control, without -turning Tempest into the active PDS for those accounts. - -- [ ] T18-01: Add a `Tempest.PersonalBackups` context and migrations for - external backup accounts, backup runs, immutable snapshots, blob records, - and retention settings. -- [ ] T18-02: Add account registration with DID, handle, source PDS URL, label, - and status fields. -- [ ] T18-03: Add identity/source verification that resolves handle and DID - document, verifies `#atproto_pds`, and fails closed on mismatched pinned - source PDS values. -- [ ] T18-04: Add credential storage for no-auth, app-password, and access-token - modes. Store secrets defensively, never render them, and allow rotation and - deletion. -- [ ] T18-05: Add a source PDS client using `Req` for `com.atproto.sync.getRepo`, - `com.atproto.sync.listBlobs`, `com.atproto.sync.getBlob`, and - `app.bsky.actor.getPreferences`. -- [ ] T18-06: Add CAR snapshot creation that stores `repo.car`, commit CID, rev, - byte size, hash, source PDS, handle, and DID. -- [ ] T18-07: Reuse repo-core verification to validate commit DID, commit - signature, MST completeness, record paths, record CIDs, and CAR integrity. -- [ ] T18-08: Extract blob references from repo records and merge them with - paginated `listBlobs` output. -- [ ] T18-09: Add bounded concurrent blob download with CID verification, - missing-blob recording, and retry handling for transient source failures. -- [ ] T18-10: Add credentialed private preference backup through - `app.bsky.actor.getPreferences`, with auth failures reported separately - from public repo/blob backup status. -- [ ] T18-11: Write immutable snapshot manifests and verification reports, first - to a temporary workspace and then atomically mark snapshots complete. -- [ ] T18-12: Store personal backup snapshots through the existing local and - S3/R2 backup storage shape. -- [ ] T18-13: Add retention policies: keep all, keep last N, keep for days, and - pinned snapshots. -- [ ] T18-14: Add portable export bundle creation containing manifest, repo CAR, blobs, - preferences JSON when present, and verification report. -- [ ] T18-15: Add offline snapshot verification that can run without contacting - the source PDS. -- [ ] T18-16: Add tests proving a snapshot can be understood from its manifest - and files without relying on Tempest database rows. -- [ ] T18-17: Add Mix tasks for backup, verify, list snapshots, export bundle, - prune, and show account backup status. -- [ ] T18-18: Add admin-only routes and controllers for external backup account - list, detail, create, edit, delete, backup now, verify, prune, and export. -- [ ] T18-19: Add admin templates under the existing UI language for credential - state, latest backup status, missing blobs, snapshot history, storage - totals, and source identity warnings. -- [ ] T18-20: Add manual backup locking so two runs cannot mutate the same - account snapshot workspace at the same time. -- [ ] T18-21: Add optional scheduled backups after manual backups are stable, - with one-account-at-a-time default scheduling and persisted run state. -- [ ] T18-22: Add unit tests for account registration, credential redaction, - manifest writing, retention pruning, and snapshot state transitions. -- [ ] T18-23: Add fixture-server integration tests for `getRepo`, `listBlobs`, - `getBlob`, preferences auth success, preferences auth failure, missing - blobs, bad CIDs, bad CARs, and source identity mismatch. -- [ ] T18-24: Add admin ConnCase tests for route auth, create/edit forms, - backup-now action, verification action, export action, and deletion - confirmation. -- [ ] T18-25: Add Hurl smoke test `test/smoke/personal-backups.hurl`. -- [ ] T18-26: Add reference documentation after implementation describing the - backup format, restore/export limits, security model, and operational - checks. - -## Integration Tests - -- Public repo backup succeeds without credentials. -- Blob backup stores every available listed or referenced blob. -- CID mismatch marks a snapshot failed. -- Missing blobs are recorded and visible to the operator. -- Private preferences are included only when valid credentials are configured. -- Auth failures do not leak secrets and do not destroy public backup output. -- Offline verification catches corrupted CAR, blob, manifest, and preferences - files. -- Retention pruning never deletes pinned snapshots. -- Admin routes reject unauthenticated, account-token, and normal app-password - requests. - -## HTTP Verification - -```bash -hurl --test --jobs 1 \ - --variable base_url=http://localhost:4000 \ - --variable admin_token="$ADMIN_TOKEN" \ - test/smoke/personal-backups.hurl -``` - -## Implementation Notes - -Keep this feature read-only against source PDS instances in the first version. -The backup client may authenticate to read private preferences, but it must not -write records, submit PLC operations, activate accounts, deactivate accounts, or -call migration endpoints. - -Prefer portable snapshot bundles over direct restore. Direct restore into Tempest -belongs in migration work after backup verification has real use. diff --git a/docs/tasks/README.md b/docs/tasks/README.md index 70ff9fe..933ed91 100644 --- a/docs/tasks/README.md +++ b/docs/tasks/README.md @@ -22,7 +22,7 @@ title: Milestone Tasks 16. [Deployment and Post-deployment Verification](./15-deployment-verification.md) 17. [Public Stats Dashboard](./16-public-stats-dashboard.md) 18. [Doc Viewer](./17-doc-viewer.md) -19. [Personal Account Backups](./18-personal-backups.md) +19. [Account Management Control Panel](./18-account-management.md) Each file in this directory is a milestone. Each task is intended to be the smallest useful unit of work: one focused implementation change, test, or