diff --git a/docs/design/spb-l2-keys-destination.md b/docs/design/spb-l2-keys-destination.md new file mode 100644 index 000000000..e394aedbd --- /dev/null +++ b/docs/design/spb-l2-keys-destination.md @@ -0,0 +1,405 @@ +# SPB L2 keys and destination + +## Gate verdict + +The L2 design is technically feasible with no blocking issue found. The only +gate decision points to carry forward are: + +- Recovery-key confirmation deliberately accepts Crockford lookalikes + (`O -> 0`, `I/L -> 1`) and ignores non-alphabet grouping/noise. This is a + usability expansion beyond literal case/whitespace matching. +- `restic cat config` on 0.19.0 does not surface normal shared repository locks: + it returned 0 while a `backup --stdin` process held a lock. Mapping returncode + 11 to `locked` is still correct and harmless because restic emits 11 for + commands that contend for exclusive locks, but tests should not require local + `cat config` to produce 11. +- Native B2 is supported by restic and is feasible, but the restic 0.19.0 docs + recommend B2's S3-compatible API for better behavior. Keeping native `b2` is a + product choice, not a technical blocker. + +External references used for destination validation: + +- Restic 0.19.0 repository/password automation, S3, MinIO, S3-compatible, and + B2 examples: https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html +- Cloudflare R2 S3 endpoint and access-key model: + https://developers.cloudflare.com/r2/get-started/s3/ + +## Module layout + +Add four new modules under `solstone/think/backup/`: + +- `keys.py`: pure crypto/string helpers. No restic, no journal config imports. +- `destination.py`: destination model, backend env assembly, sanitized read-only + reachability probe. +- `repo.py`: repository initialization and recovery-key installation via restic. +- `state.py`: config-section accessors using only `solstone.think.journal_config` + helpers for reads/writes/locking. + +Also make these minimal edits: + +- `runner.py`: add optional `pass_fds` support and thread it to + `subprocess.run`. +- `journal_default.json`: add the top-level `backup` schema. + +No CLI, UI, scheduler, execution loop, backup command, or sol-pbc service contact +is in this lode. + +## Durable contracts + +### Recovery keys + +Canonical recovery key: + +- Exactly 64 uppercase Crockford characters. +- Alphabet source: `from solstone.apps.link.crockford32 import ALPHABET`. +- Generated as `secrets.choice(ALPHABET)` per character. +- 64 characters * 5 bits per Crockford character = 320 bits. +- Persist the canonical form. + +Display recovery key: + +- 16 groups of 4 canonical characters. +- Groups are joined by one space. +- `format_recovery_key_display(canonical) -> str` is the only formatter. + +Normalization and confirmation: + +- `normalize_recovery_key(value)` uppercases, folds `I` and `L` to `1`, folds + `O` to `0`, then drops characters not in `ALPHABET`. +- `confirm_recovery_key(candidate, canonical)` compares normalized strings for + exact equality. +- Do not use constant-time comparison. The key is owner-entered local setup + material, not an online oracle. +- Do not import `crockford32._normalize_char`; keep the recovery-key normalizer + self-contained. + +False-accept proof for lookalike folding: + +- Canonical keys are drawn only from `ALPHABET`, which excludes `I`, `L`, `O`, + and `U`. +- Folding `I/L` can only produce `1`; folding `O` can only produce `0`. +- Because canonical keys never contain the folded source characters, two + different canonical keys cannot collapse to the same normalized form through + these folds. +- A candidate with a folded character is accepted only when it normalizes to the + exact canonical key. If it differs at any real canonical position, equality + fails. + +Alphabet import validation: + +- `solstone/apps/link/crockford32.py` imports no solstone modules. It defines + `ALPHABET` at line 8 and depends only on local constants/functions, so importing + the constant from `keys.py` cannot create a cycle. +- The same think-to-apps edge already exists in + `solstone/think/link/join_cli.py:49`. +- Factoring one constant into a new think module would broaden the lode. + +### Backup config + +Add this top-level `backup` section to `solstone/think/journal_default.json`: + +- `enabled`: `false` +- `mode`: `"byo"` +- `destination`: `repository: null`, `backend: null`, `credentials: {}` +- `daily_key`: `null` +- `recovery_key`: `null` +- `confirmed_recovery_key`: `false` +- `retention`: `hourly: 24`, `daily: 7`, `weekly: 4`, `monthly: 12` +- `schedule`: `every: "daily"`, `enabled: false` +- `last_backup`: `time: null`, `snapshot_id: null`, `status: null`, + `error_reason: null` + +`last_backup` is schema-only in L2. Do not add an unused update setter. + +Existing journals do not get default merges. `state.py` readers must default +per field when reading an existing partial `journal.json`. + +Power-user raw-password escape: + +- `generate_and_store_keys()` is get-or-create. +- If `backup.daily_key` is non-null, respect it exactly and do not regenerate. +- If `backup.recovery_key` is non-null, respect it exactly and do not regenerate. +- If missing, generate `daily_key` with `secrets.token_urlsafe(32)` and + `recovery_key` with the canonical Crockford generator. +- Detection rule is simple non-null presence in config. + +### Destination config + +`Destination(repository: str, backend: str, credentials: dict[str, str])` +contains: + +- `repository`: credential-free restic repository string, passed directly as + `RESTIC_REPOSITORY`. +- `backend`: discriminator, one of `"s3"` or `"b2"`. +- `credentials`: backend-specific secrets, passed only through environment. + +Credential env assembly: + +- `s3`: `AWS_ACCESS_KEY_ID` from `access_key_id`, `AWS_SECRET_ACCESS_KEY` from + `secret_access_key`. +- `b2`: `B2_ACCOUNT_ID` from `account_id`, `B2_ACCOUNT_KEY` from `account_key`. +- Unknown backend or missing required credential fields raises a clear boundary + error. + +Canonical credential-free repository examples: + +- AWS S3: `s3:s3.us-east-1.amazonaws.com/bucket_name/path/to/repo` +- Cloudflare R2: `s3:https://.r2.cloudflarestorage.com/bucket_name/path` +- MinIO or S3-compatible: `s3:http://localhost:9000/bucket_name/path` +- Native Backblaze B2: `b2:bucketname:path/to/repo` + +Do not embed access keys, secret keys, session tokens, or presigned URLs in the +repository string. + +## Restic invocation model + +`run_restic` remains the only restic invocation path in source code. + +Runner extension: + +- Add `pass_fds: tuple[int, ...] = ()` to `run_restic`. +- Thread it to `subprocess.run(..., pass_fds=pass_fds)`. +- Default empty tuple preserves all existing callers. +- Python subprocess supports `pass_fds` with `close_fds=True`; supplying + `pass_fds` forces `close_fds` behavior for every other descriptor. This matches + the desired pipe-only inheritance model. + +Two-key install: + +- Initialize with the daily key as `RESTIC_PASSWORD` through `run_restic`. +- Add the recovery key with `restic key add --new-password-file /dev/fd/`. +- Existing repo unlock for `key add` still uses daily key in `RESTIC_PASSWORD`. +- The recovery key is delivered via an `os.pipe()` read end passed with + `pass_fds`. + +Pipe protocol: + +- Write `recovery_key + "\n"` to the pipe write end. +- The payload is about 65 bytes, far below Linux pipe capacity and POSIX + `PIPE_BUF`, so the pre-spawn write is atomic and cannot fill the pipe. +- Close the write end before spawning restic. +- Spawn `run_restic(..., pass_fds=(read_fd,))`. +- Restic opens `/dev/fd/`, reads the buffered bytes, sees EOF, and + exits. +- Parent closes all pipe fds in `finally`. +- No writer thread is needed. + +Guard/scrub interaction: + +- The recovery key never appears in argv. Argv contains only `/dev/fd/`. +- The recovery key is not the `password` argument and is not in `backend_env`, so + current scrub logic will not scrub it. +- Empirical 0.19.0 `key add` does not echo the new password. +- Do not add an extra scrub-secrets parameter in L2. If future restic output ever + echoes the new password, adding extra scrub secrets is a small runner extension. + +## Repository init state machine + +The repo itself is the source of truth. There is no persisted initialized flag. + +`init_repository(destination, daily_key, recovery_key, restic_path, timeout=None)` +does: + +1. `validate_destination(destination, daily_key)`. +2. If `repo_missing`, run `restic init` with the daily key, add recovery key, then + verify recovery unlocks. +3. If `repo_exists`, probe with the recovery key. +4. If recovery also returns `repo_exists`, no-op. +5. If recovery returns `auth_failed`, add the recovery key with daily unlock, then + verify recovery unlocks. +6. If daily returns `auth_failed`, raise conflict: repo exists but our daily key + does not unlock it. Do not init or overwrite. +7. If status is `locked`, `timeout`, or `unreachable`, raise or return a clear + setup error. Do not init. + +Partial-failure coverage: + +- Init succeeds, recovery add fails: rerun sees daily unlock and recovery + `auth_failed`, then adds recovery. +- Recovery add succeeds, final verify is interrupted: rerun sees both keys unlock + and no-ops. +- Destination existed before setup with another password: daily probe returns + `auth_failed`; code raises conflict and never reinitializes. +- Re-init is avoided because restic returns 1 on an already initialized repo. + +## Sanitized destination validation + +`validate_destination` runs `restic cat config` through `run_restic` and never +returns or logs raw stdout/stderr. + +Return dataclass: + +- `DestinationStatus(reachable: bool, repo_exists: bool, reason_code: str, message: str)` + +Reason-code mapping: + +- `0`: `repo_exists`, reachable true, repo_exists true, + message `backup repository is reachable` +- `10`: `repo_missing`, reachable true, repo_exists false, + message `backup destination is reachable and needs setup` +- `12`: `auth_failed`, reachable true, repo_exists true, + message `repository password was rejected` +- `11`: `locked`, reachable true, repo_exists true, + message `repository is locked; try again shortly` +- `124`: `timeout`, reachable false, repo_exists false, + message `could not reach the backup destination` +- Any other nonzero: `unreachable`, reachable false, repo_exists false, + message `could not reach the backup destination` + +Logging, if any, is limited to returncode and reason code. + +## Function and signature inventory + +### `keys.py` + +- `generate_daily_key() -> str` + Pure secret generator. No journal mutation. +- `generate_recovery_key() -> str` + Pure canonical recovery-key generator. No journal mutation. +- `format_recovery_key_display(canonical: str) -> str` + Read/format helper. Validates canonical length/alphabet. +- `normalize_recovery_key(value: str) -> str` + Read/parse helper. Applies folding and grouping/noise removal. +- `confirm_recovery_key(candidate: str, canonical: str) -> bool` + Read/compare helper. + +### `destination.py` + +- `Destination(repository: str, backend: str, credentials: dict[str, str])` +- `DestinationStatus(reachable: bool, repo_exists: bool, reason_code: str, message: str)` +- `assemble_backend_env(destination: Destination) -> dict[str, str]` + Read/assembly helper. No journal mutation. +- `validate_destination(destination: Destination, password: str, *, restic_path: Path, timeout: float | None = None) -> DestinationStatus` + Read-only restic probe. Sanitizes all output. + +### `repo.py` + +- `init_repository(destination: Destination, *, daily_key: str, recovery_key: str, restic_path: Path, timeout: float | None = None) -> None` + Write verb. May initialize remote repo and add restic key. +- `_add_recovery_key(destination: Destination, *, daily_key: str, recovery_key: str, restic_path: Path, timeout: float | None = None) -> None` + Internal write helper for `restic key add` via pipe-FD. +- `_verify_recovery_key(destination: Destination, *, recovery_key: str, restic_path: Path, timeout: float | None = None) -> None` + Internal read/guard helper. Raises on failed verification. + +### `state.py` + +- `BackupKeys(daily_key: str, recovery_key: str, recovery_key_display: str)` +- `get_backup_config() -> dict[str, Any]` + Read accessor with per-field defaults. +- `get_destination() -> Destination | None` + Read accessor. Returns `None` until repository/backend are set. +- `get_keys() -> BackupKeys | None` + Read accessor. Returns `None` until both keys are present. +- `generate_and_store_keys() -> BackupKeys` + Write accessor. Uses `hold_config_lock`, preserves non-null existing keys. +- `set_destination(destination: Destination) -> None` + Write accessor. Uses `hold_config_lock`. +- `set_recovery_key_confirmed(confirmed: bool = True) -> None` + Write accessor. Uses `hold_config_lock`. +- `status_view() -> dict[str, Any]` + Read accessor. Redacted owner-safe state only. + +### `runner.py` + +- `run_restic(..., pass_fds: tuple[int, ...] = ()) -> ResticResult` + Existing behavior with optional FD inheritance for pipe-backed password files. + +Read/write naming: + +- Read/no journal mutation: `get_`, `validate_`, `assemble_`, `format_`, + `normalize_`, `confirm_`. +- Write/mutating: `generate_and_store_`, `set_`, `init_`, `_add_`. + +## Redacted status view + +`status_view()` returns only: + +- `enabled` +- `mode` +- `destination`: `repository`, `backend`, `credentials_set` +- `daily_key_set` +- `recovery_key_set` +- `recovery_key_confirmed` +- `retention` +- `schedule` +- `last_backup` + +No `daily_key`, `recovery_key`, or credential values are present. Credentials +collapse to a boolean. Repository is shown because it is owner destination +identity, not the credential channel. + +## Test plan + +Add focused unit tests: + +- `tests/test_backup_keys.py` + - canonical key length/alphabet. + - display grouping is 16 groups of 4. + - normalization accepts spaces, hyphens, case, and lookalikes. + - lookalike folds do not collapse different canonical keys. + - invalid canonical formatting raises loudly. +- `tests/test_backup_destination.py` + - s3 env assembly. + - b2 env assembly. + - unknown backend and missing credentials fail loudly. + - `validate_destination` maps returncodes 0, 10, 11, 12, 124, and unknown. + - raw restic stderr/stdout never appears in returned status or logs. +- `tests/test_backup_state.py` + - default backup section materializes by accessor when absent. + - existing partial `journal.json` gets per-field defaults on read. + - setters use `write_journal_config` under config lock. + - `generate_and_store_keys` preserves hand-set daily key. + - status view contains no key or credential values. +- `tests/test_backup_runner.py` + - pass_fds default is `()`. + - pass_fds is threaded to monkeypatched `subprocess.run`. + - existing scrub/argv/env tests remain unchanged. +- Optional local integration test, skipped unless `shutil.which("restic")` exists: + - initialize a local repo. + - add recovery key through pipe-FD. + - prove daily and recovery keys independently unlock. + - No network required. + +The existing host-restic integration idiom can be mirrored, but all behavior +that depends on 0.19.0 semantics should use the vendored path in implementation +tests where practical. + +## Hygiene checklist + +- SPDX headers and `from __future__ import annotations` on new Python files. +- `state.py` imports `journal_config` helpers only; no `journal_io` primitives. +- Config writes go through `write_journal_config`. +- Read-modify-write setters hold `hold_config_lock`. +- No `os.replace`, temp `.replace`, or custom atomic write code in backup state. +- No layer-hygiene allowlist edits expected. +- No sol-pbc endpoint, support endpoint, portal endpoint, or relay endpoint. +- Restic raw stderr/stdout is never owner-visible from destination validation. +- Recovery key and backend credentials never appear in argv. + +## Implementation sequence + +1. Extend `runner.py` with `pass_fds` and tests. +2. Add `keys.py` and key tests. +3. Add `destination.py` and sanitized mapping tests. +4. Add `state.py`, default schema, and redaction/config tests. +5. Add `repo.py` and idempotency tests using monkeypatched + `validate_destination`/`run_restic`. +6. Add optional local restic integration test. +7. Run focused tests, then `make ci` before commit. + +## Irreversible owner-facing commitments + +Once an owner records recovery keys or initializes a repository: + +- The recovery key is a durable master key. Losing both daily and recovery keys + makes the backup data irrecoverable. +- The canonical recovery key written to paper must continue to unlock the repo. + Silent regeneration is forbidden. +- Repository initialization writes real restic config/key material into the + owner-selected destination. Reinitializing the same path is not a repair path. +- The daily key in config is part of the repo key set. Hand-editing it later does + not change the repository key; it only changes what solstone will try. +- The selected repository string points at external owner storage. Deleting or + reusing that path outside solstone can destroy or conflict with backup state. +- The repo format/chunker parameters are fixed by restic at init and become part + of future compatibility expectations. diff --git a/solstone/think/backup/destination.py b/solstone/think/backup/destination.py new file mode 100644 index 000000000..3512d381b --- /dev/null +++ b/solstone/think/backup/destination.py @@ -0,0 +1,129 @@ +# SPDX-License-Identifier: AGPL-3.0-only +# Copyright (c) 2026 sol pbc + +"""Backup destination model, backend credentials, and sanitized probes.""" + +from __future__ import annotations + +import logging +from dataclasses import dataclass +from pathlib import Path + +from solstone.think.backup.runner import run_restic + +logger = logging.getLogger("solstone.backup.destination") + + +@dataclass(frozen=True) +class Destination: + repository: str + backend: str + credentials: dict[str, str] + + +@dataclass(frozen=True) +class DestinationStatus: + reachable: bool + repo_exists: bool + reason_code: str + message: str + + +_STATUS_BY_RETURNCODE = { + 0: DestinationStatus( + reachable=True, + repo_exists=True, + reason_code="repo_exists", + message="backup repository is reachable", + ), + 10: DestinationStatus( + reachable=True, + repo_exists=False, + reason_code="repo_missing", + message="backup destination is reachable and needs setup", + ), + 12: DestinationStatus( + reachable=True, + repo_exists=True, + reason_code="auth_failed", + message="repository password was rejected", + ), + 11: DestinationStatus( + reachable=True, + repo_exists=True, + reason_code="locked", + message="repository is locked; try again shortly", + ), + 124: DestinationStatus( + reachable=False, + repo_exists=False, + reason_code="timeout", + message="could not reach the backup destination", + ), +} +_UNREACHABLE_STATUS = DestinationStatus( + reachable=False, + repo_exists=False, + reason_code="unreachable", + message="could not reach the backup destination", +) + + +def _require_credential(credentials: dict[str, str], key: str) -> str: + try: + return credentials[key] + except KeyError as exc: + raise KeyError(f"missing backup credential: {key}") from exc + + +def assemble_backend_env(destination: Destination) -> dict[str, str]: + credentials = destination.credentials + if destination.backend == "s3": + return { + "AWS_ACCESS_KEY_ID": _require_credential( + credentials, + "access_key_id", + ), + "AWS_SECRET_ACCESS_KEY": _require_credential( + credentials, + "secret_access_key", + ), + } + if destination.backend == "b2": + return { + "B2_ACCOUNT_ID": _require_credential(credentials, "account_id"), + "B2_ACCOUNT_KEY": _require_credential(credentials, "account_key"), + } + raise ValueError(f"unsupported backup backend: {destination.backend!r}") + + +def validate_destination( + destination: Destination, + password: str, + *, + restic_path: Path, + timeout: float | None = None, +) -> DestinationStatus: + result = run_restic( + ["cat", "config"], + repository=destination.repository, + password=password, + restic_path=restic_path, + backend_env=assemble_backend_env(destination), + timeout=timeout, + ) + status = _STATUS_BY_RETURNCODE.get(result.returncode, _UNREACHABLE_STATUS) + logger.debug( + "backup destination probe completed returncode=%s reason_code=%s", + result.returncode, + status.reason_code, + ) + return status + + +__all__ = [ + "Destination", + "DestinationStatus", + "assemble_backend_env", + "validate_destination", +] diff --git a/solstone/think/backup/keys.py b/solstone/think/backup/keys.py new file mode 100644 index 000000000..3f7a670b2 --- /dev/null +++ b/solstone/think/backup/keys.py @@ -0,0 +1,76 @@ +# SPDX-License-Identifier: AGPL-3.0-only +# Copyright (c) 2026 sol pbc + +"""Pure key generation and recovery-key formatting for sol private backup.""" + +from __future__ import annotations + +import secrets + +from solstone.apps.link.crockford32 import ALPHABET + +RECOVERY_KEY_LENGTH = 64 +_RECOVERY_GROUP_SIZE = 4 +_ALPHABET_SET = frozenset(ALPHABET) +_LOOKALIKE_FOLDS = { + "I": "1", + "L": "1", + "O": "0", +} + + +def generate_daily_key() -> str: + return secrets.token_urlsafe(32) + + +def generate_recovery_key() -> str: + return "".join(secrets.choice(ALPHABET) for _ in range(RECOVERY_KEY_LENGTH)) + + +def _validate_canonical_recovery_key(canonical: str) -> None: + if len(canonical) != RECOVERY_KEY_LENGTH: + raise ValueError("canonical recovery key must be exactly 64 characters") + if any(char not in _ALPHABET_SET for char in canonical): + raise ValueError("canonical recovery key contains invalid Crockford characters") + + +def format_recovery_key_display(canonical: str) -> str: + _validate_canonical_recovery_key(canonical) + return " ".join( + canonical[index : index + _RECOVERY_GROUP_SIZE] + for index in range(0, RECOVERY_KEY_LENGTH, _RECOVERY_GROUP_SIZE) + ) + + +def parse_recovery_key(entered: str) -> str: + chars: list[str] = [] + for raw_char in entered: + char = raw_char.upper() + char = _LOOKALIKE_FOLDS.get(char, char) + if char in _ALPHABET_SET: + chars.append(char) + + canonical = "".join(chars) + if len(canonical) != RECOVERY_KEY_LENGTH: + raise ValueError( + "recovery key must contain exactly 64 Crockford characters after cleanup" + ) + return canonical + + +def confirm_recovery_key(entered: str, canonical: str) -> bool: + try: + return parse_recovery_key(entered) == canonical + except ValueError: + return False + + +__all__ = [ + "ALPHABET", + "RECOVERY_KEY_LENGTH", + "confirm_recovery_key", + "format_recovery_key_display", + "generate_daily_key", + "generate_recovery_key", + "parse_recovery_key", +] diff --git a/solstone/think/backup/repo.py b/solstone/think/backup/repo.py new file mode 100644 index 000000000..122e68744 --- /dev/null +++ b/solstone/think/backup/repo.py @@ -0,0 +1,152 @@ +# SPDX-License-Identifier: AGPL-3.0-only +# Copyright (c) 2026 sol pbc + +"""Restic repository initialization for sol private backup.""" + +from __future__ import annotations + +import os +from pathlib import Path + +from solstone.think.backup.destination import ( + Destination, + assemble_backend_env, + validate_destination, +) +from solstone.think.backup.runner import run_restic + + +def init_repository( + destination: Destination, + *, + daily_key: str, + recovery_key: str, + restic_path: Path, + timeout: float | None = None, +) -> None: + status = validate_destination( + destination, + daily_key, + restic_path=restic_path, + timeout=timeout, + ) + if status.reason_code == "repo_missing": + result = run_restic( + ["init"], + repository=destination.repository, + password=daily_key, + restic_path=restic_path, + backend_env=assemble_backend_env(destination), + timeout=timeout, + ) + if result.returncode != 0: + raise RuntimeError( + f"restic init failed with returncode {result.returncode}" + ) + _add_recovery_key( + destination, + daily_key=daily_key, + recovery_key=recovery_key, + restic_path=restic_path, + timeout=timeout, + ) + _verify_recovery_key( + destination, + recovery_key=recovery_key, + restic_path=restic_path, + timeout=timeout, + ) + return + + if status.reason_code == "repo_exists": + recovery_status = validate_destination( + destination, + recovery_key, + restic_path=restic_path, + timeout=timeout, + ) + if recovery_status.repo_exists and recovery_status.reason_code == "repo_exists": + return + if recovery_status.reason_code == "auth_failed": + _add_recovery_key( + destination, + daily_key=daily_key, + recovery_key=recovery_key, + restic_path=restic_path, + timeout=timeout, + ) + _verify_recovery_key( + destination, + recovery_key=recovery_key, + restic_path=restic_path, + timeout=timeout, + ) + return + raise RuntimeError(recovery_status.message) + + if status.reason_code == "auth_failed": + raise RuntimeError( + "backup repository exists but the configured daily key does not unlock it" + ) + + raise RuntimeError(status.message) + + +def _add_recovery_key( + destination: Destination, + *, + daily_key: str, + recovery_key: str, + restic_path: Path, + timeout: float | None = None, +) -> None: + read_fd = -1 + write_fd = -1 + try: + read_fd, write_fd = os.pipe() + payload = (recovery_key + "\n").encode() + os.write(write_fd, payload) + os.close(write_fd) + write_fd = -1 + result = run_restic( + ["key", "add", "--new-password-file", f"/dev/fd/{read_fd}"], + repository=destination.repository, + password=daily_key, + restic_path=restic_path, + backend_env=assemble_backend_env(destination), + timeout=timeout, + pass_fds=(read_fd,), + ) + finally: + for fd in (write_fd, read_fd): + if fd == -1: + continue + try: + os.close(fd) + except OSError: + pass + + if result.returncode != 0: + raise RuntimeError(f"restic key add failed with returncode {result.returncode}") + + +def _verify_recovery_key( + destination: Destination, + *, + recovery_key: str, + restic_path: Path, + timeout: float | None = None, +) -> None: + status = validate_destination( + destination, + recovery_key, + restic_path=restic_path, + timeout=timeout, + ) + if not (status.repo_exists and status.reason_code == "repo_exists"): + raise RuntimeError("recovery key did not unlock the repository after key add") + + +__all__ = [ + "init_repository", +] diff --git a/solstone/think/backup/runner.py b/solstone/think/backup/runner.py index 64fd3bb3f..0f8493245 100644 --- a/solstone/think/backup/runner.py +++ b/solstone/think/backup/runner.py @@ -116,6 +116,7 @@ def run_restic( json: bool = False, max_repack_size: str | None = None, timeout: float | None = None, + pass_fds: tuple[int, ...] = (), ) -> ResticResult: env, secrets = _child_env(repository, password, backend_env) argv = _build_argv(restic_path, args, json, max_repack_size) @@ -133,6 +134,7 @@ def run_restic( text=True, env=env, timeout=timeout, + pass_fds=pass_fds, ) except subprocess.TimeoutExpired as exc: stdout = _scrub(_timeout_text(exc.stdout), secrets) diff --git a/solstone/think/backup/state.py b/solstone/think/backup/state.py new file mode 100644 index 000000000..e905a7642 --- /dev/null +++ b/solstone/think/backup/state.py @@ -0,0 +1,191 @@ +# SPDX-License-Identifier: AGPL-3.0-only +# Copyright (c) 2026 sol pbc + +"""Journal config accessors for sol private backup state.""" + +from __future__ import annotations + +import copy +from dataclasses import dataclass +from typing import Any + +from solstone.think.backup.destination import Destination +from solstone.think.backup.keys import ( + format_recovery_key_display, + generate_daily_key, + generate_recovery_key, +) +from solstone.think.journal_config import ( + hold_config_lock, + read_journal_config, + write_journal_config, +) + +BACKUP_DEFAULTS: dict[str, Any] = { + "enabled": False, + "mode": "byo", + "destination": { + "repository": None, + "backend": None, + "credentials": {}, + }, + "daily_key": None, + "recovery_key": None, + "confirmed_recovery_key": False, + "retention": { + "hourly": 24, + "daily": 7, + "weekly": 4, + "monthly": 12, + }, + "schedule": { + "every": "daily", + "enabled": False, + }, + "last_backup": { + "time": None, + "snapshot_id": None, + "status": None, + "error_reason": None, + }, +} + + +@dataclass(frozen=True) +class BackupKeys: + daily_key: str + recovery_key: str + recovery_key_display: str + + +def _merge_defaults(defaults: dict[str, Any], raw: Any) -> dict[str, Any]: + merged = copy.deepcopy(defaults) + if not isinstance(raw, dict): + return merged + for key, value in raw.items(): + if isinstance(merged.get(key), dict) and isinstance(value, dict): + merged[key] = _merge_defaults(merged[key], value) + else: + merged[key] = value + return merged + + +def _writable_backup_section(config: dict[str, Any]) -> dict[str, Any]: + backup = config.get("backup") + if not isinstance(backup, dict): + backup = {} + config["backup"] = backup + return backup + + +def _build_backup_keys(daily_key: Any, recovery_key: Any) -> BackupKeys | None: + if daily_key is None or recovery_key is None: + return None + if not isinstance(daily_key, str) or not isinstance(recovery_key, str): + raise ValueError("backup keys must be strings when present") + return BackupKeys( + daily_key=daily_key, + recovery_key=recovery_key, + recovery_key_display=format_recovery_key_display(recovery_key), + ) + + +def get_backup_config() -> dict[str, Any]: + config = read_journal_config() + return _merge_defaults(BACKUP_DEFAULTS, config.get("backup", {})) + + +def get_destination() -> Destination | None: + destination = get_backup_config()["destination"] + repository = destination.get("repository") + backend = destination.get("backend") + credentials = destination.get("credentials", {}) + if repository is None or backend is None: + return None + if not isinstance(repository, str) or not isinstance(backend, str): + raise ValueError("backup destination repository and backend must be strings") + if not isinstance(credentials, dict): + raise ValueError("backup destination credentials must be a JSON object") + return Destination( + repository=repository, + backend=backend, + credentials=dict(credentials), + ) + + +def get_keys() -> BackupKeys | None: + config = get_backup_config() + return _build_backup_keys(config["daily_key"], config["recovery_key"]) + + +def generate_and_store_keys() -> BackupKeys: + with hold_config_lock(): + config = read_journal_config() + backup = _writable_backup_section(config) + daily_key = backup.get("daily_key") + recovery_key = backup.get("recovery_key") + if daily_key is None: + daily_key = generate_daily_key() + if recovery_key is None: + recovery_key = generate_recovery_key() + backup["daily_key"] = daily_key + backup["recovery_key"] = recovery_key + keys = _build_backup_keys(daily_key, recovery_key) + if keys is None: + raise RuntimeError("backup key generation failed") + write_journal_config(config) + return keys + + +def set_destination(destination: Destination) -> None: + with hold_config_lock(): + config = read_journal_config() + backup = _writable_backup_section(config) + backup["destination"] = { + "repository": destination.repository, + "backend": destination.backend, + "credentials": dict(destination.credentials), + } + write_journal_config(config) + + +def set_recovery_key_confirmed(confirmed: bool = True) -> None: + with hold_config_lock(): + config = read_journal_config() + backup = _writable_backup_section(config) + backup["confirmed_recovery_key"] = confirmed + write_journal_config(config) + + +def status_view() -> dict[str, Any]: + config = get_backup_config() + destination = config["destination"] + credentials = destination.get("credentials") + return { + "enabled": config["enabled"], + "mode": config["mode"], + "destination": { + "repository": destination.get("repository"), + "backend": destination.get("backend"), + "credentials_set": bool(credentials), + }, + "daily_key_set": config["daily_key"] is not None, + "recovery_key_set": config["recovery_key"] is not None, + "recovery_key_confirmed": bool(config["confirmed_recovery_key"]), + "retention": config["retention"], + "schedule": config["schedule"], + "last_backup": config["last_backup"], + } + + +__all__ = [ + "BACKUP_DEFAULTS", + "BackupKeys", + "generate_and_store_keys", + "get_backup_config", + "get_destination", + "get_keys", + "set_destination", + "set_recovery_key_confirmed", + "status_view", +] diff --git a/solstone/think/journal_default.json b/solstone/think/journal_default.json index 9d2333be1..f84ecfc02 100644 --- a/solstone/think/journal_default.json +++ b/solstone/think/journal_default.json @@ -80,6 +80,34 @@ "pairing": { "host_url": null }, + "backup": { + "enabled": false, + "mode": "byo", + "destination": { + "repository": null, + "backend": null, + "credentials": {} + }, + "daily_key": null, + "recovery_key": null, + "confirmed_recovery_key": false, + "retention": { + "hourly": 24, + "daily": 7, + "weekly": 4, + "monthly": 12 + }, + "schedule": { + "every": "daily", + "enabled": false + }, + "last_backup": { + "time": null, + "snapshot_id": null, + "status": null, + "error_reason": null + } + }, "retention": { "raw_media": "keep", "raw_media_days": null, diff --git a/tests/test_backup_destination.py b/tests/test_backup_destination.py new file mode 100644 index 000000000..b9123046f --- /dev/null +++ b/tests/test_backup_destination.py @@ -0,0 +1,170 @@ +# SPDX-License-Identifier: AGPL-3.0-only +# Copyright (c) 2026 sol pbc + +from __future__ import annotations + +import json +import logging +from pathlib import Path +from typing import Any + +import pytest + +from solstone.think.backup import destination +from solstone.think.backup.destination import ( + Destination, + assemble_backend_env, + validate_destination, +) +from solstone.think.backup.runner import ResticResult + + +def test_assemble_backend_env_s3() -> None: + dest = Destination( + repository="s3:s3.us-east-1.amazonaws.com/bucket/path", + backend="s3", + credentials={ + "access_key_id": "access-key", + "secret_access_key": "secret-key", + }, + ) + + assert assemble_backend_env(dest) == { + "AWS_ACCESS_KEY_ID": "access-key", + "AWS_SECRET_ACCESS_KEY": "secret-key", + } + + +def test_assemble_backend_env_b2() -> None: + dest = Destination( + repository="b2:bucket:path", + backend="b2", + credentials={ + "account_id": "account-id", + "account_key": "account-key", + }, + ) + + assert assemble_backend_env(dest) == { + "B2_ACCOUNT_ID": "account-id", + "B2_ACCOUNT_KEY": "account-key", + } + + +def test_assemble_backend_env_rejects_unknown_backend() -> None: + dest = Destination(repository="repo", backend="unknown", credentials={}) + + with pytest.raises(ValueError, match="unsupported backup backend"): + assemble_backend_env(dest) + + +def test_assemble_backend_env_rejects_missing_credentials() -> None: + dest = Destination( + repository="s3:s3.us-east-1.amazonaws.com/bucket/path", + backend="s3", + credentials={"access_key_id": "access-key"}, + ) + + with pytest.raises(KeyError, match="secret_access_key"): + assemble_backend_env(dest) + + +@pytest.mark.parametrize( + ("returncode", "reason_code", "reachable", "repo_exists", "message"), + [ + (0, "repo_exists", True, True, "backup repository is reachable"), + ( + 10, + "repo_missing", + True, + False, + "backup destination is reachable and needs setup", + ), + (11, "locked", True, True, "repository is locked; try again shortly"), + (12, "auth_failed", True, True, "repository password was rejected"), + (124, "timeout", False, False, "could not reach the backup destination"), + (77, "unreachable", False, False, "could not reach the backup destination"), + ], +) +def test_validate_destination_maps_sanitized_status( + monkeypatch: pytest.MonkeyPatch, + caplog: pytest.LogCaptureFixture, + returncode: int, + reason_code: str, + reachable: bool, + repo_exists: bool, + message: str, +) -> None: + raw_secret = "presigned-url-signature" + + def fake_run_restic(args: list[str], **kwargs: Any) -> ResticResult: + return ResticResult( + returncode=returncode, + stdout=f"raw stdout {raw_secret}", + stderr=f"raw stderr {raw_secret}", + json=None, + argv=("restic", *args), + ) + + monkeypatch.setattr(destination, "run_restic", fake_run_restic) + caplog.set_level(logging.DEBUG, logger="solstone.backup.destination") + dest = Destination( + repository="s3:safe-bucket/path", + backend="s3", + credentials={ + "access_key_id": "access-key", + "secret_access_key": "secret-key", + }, + ) + + status = validate_destination( + dest, + "repo-password", + restic_path=Path("/usr/bin/restic"), + ) + + assert status.reason_code == reason_code + assert status.reachable is reachable + assert status.repo_exists is repo_exists + assert status.message == message + serialized = json.dumps(status.__dict__) + assert raw_secret not in serialized + assert raw_secret not in caplog.text + + +def test_validate_destination_passes_repo_and_creds_separately( + monkeypatch: pytest.MonkeyPatch, +) -> None: + captured: dict[str, Any] = {} + + def fake_run_restic(args: list[str], **kwargs: Any) -> ResticResult: + captured["args"] = args + captured.update(kwargs) + return ResticResult( + returncode=0, + stdout="", + stderr="", + json=None, + argv=("restic", *args), + ) + + monkeypatch.setattr(destination, "run_restic", fake_run_restic) + dest = Destination( + repository="s3:https://account.r2.cloudflarestorage.com/bucket/path", + backend="s3", + credentials={ + "access_key_id": "access-key", + "secret_access_key": "secret-key", + }, + ) + + validate_destination(dest, "repo-password", restic_path=Path("/usr/bin/restic")) + + assert captured["args"] == ["cat", "config"] + assert captured["repository"] == dest.repository + assert "access-key" not in captured["repository"] + assert "secret-key" not in captured["repository"] + assert captured["backend_env"] == { + "AWS_ACCESS_KEY_ID": "access-key", + "AWS_SECRET_ACCESS_KEY": "secret-key", + } diff --git a/tests/test_backup_keys.py b/tests/test_backup_keys.py new file mode 100644 index 000000000..eed8c5d42 --- /dev/null +++ b/tests/test_backup_keys.py @@ -0,0 +1,120 @@ +# SPDX-License-Identifier: AGPL-3.0-only +# Copyright (c) 2026 sol pbc + +from __future__ import annotations + +import pytest + +from solstone.think.backup import keys +from solstone.think.backup.keys import ( + ALPHABET, + RECOVERY_KEY_LENGTH, + confirm_recovery_key, + format_recovery_key_display, + generate_daily_key, + generate_recovery_key, + parse_recovery_key, +) + + +def test_generate_daily_key_uses_urlsafe_token(monkeypatch: pytest.MonkeyPatch) -> None: + values = iter(["daily-one", "daily-two"]) + calls: list[int] = [] + + def fake_token_urlsafe(length: int) -> str: + calls.append(length) + return next(values) + + monkeypatch.setattr(keys.secrets, "token_urlsafe", fake_token_urlsafe) + + assert generate_daily_key() == "daily-one" + assert generate_daily_key() == "daily-two" + assert calls == [32, 32] + + +def test_generate_recovery_key_uses_crockford_choice_per_char( + monkeypatch: pytest.MonkeyPatch, +) -> None: + calls = 0 + + def fake_choice(alphabet: str) -> str: + nonlocal calls + calls += 1 + assert alphabet == ALPHABET + return "A" if calls <= RECOVERY_KEY_LENGTH else "B" + + monkeypatch.setattr(keys.secrets, "choice", fake_choice) + + first = generate_recovery_key() + second = generate_recovery_key() + + assert first == "A" * RECOVERY_KEY_LENGTH + assert second == "B" * RECOVERY_KEY_LENGTH + assert calls == RECOVERY_KEY_LENGTH * 2 + + +def test_recovery_key_display_groups_canonical_key() -> None: + canonical = "ABCDEFGHJKMNPQRSTVWXYZ0123456789" * 2 + + display = format_recovery_key_display(canonical) + + groups = display.split(" ") + assert len(groups) == 16 + assert all(len(group) == 4 for group in groups) + assert "".join(groups) == canonical + + +@pytest.mark.parametrize( + "canonical", + [ + "A" * 63, + "A" * 65, + "U" * 64, + "a" * 64, + ], +) +def test_recovery_key_display_rejects_invalid_canonical(canonical: str) -> None: + with pytest.raises(ValueError): + format_recovery_key_display(canonical) + + +def test_parse_recovery_key_strips_grouping_and_uppercases() -> None: + canonical = "ABCDEFGHJKMNPQRSTVWXYZ0123456789" * 2 + entered = ( + f"{canonical[:8].lower()} - {canonical[8:16]}\n" + f"{canonical[16:32].lower()} {canonical[32:]}" + ) + + assert parse_recovery_key(entered) == canonical + + +def test_parse_recovery_key_folds_lookalikes_and_confirm_accepts() -> None: + canonical = "011" + ("A" * 61) + entered = "OIL " + " ".join( + canonical[index : index + 4] for index in range(3, RECOVERY_KEY_LENGTH, 4) + ) + + assert parse_recovery_key(entered) == canonical + assert confirm_recovery_key(entered, canonical) is True + + +@pytest.mark.parametrize("entered", ["", "not-a-key", "A" * 63, "A" * 65]) +def test_parse_recovery_key_rejects_wrong_cleaned_length(entered: str) -> None: + with pytest.raises(ValueError, match="exactly 64"): + parse_recovery_key(entered) + + +def test_confirm_recovery_key_rejects_wrong_key_without_raising() -> None: + canonical = "A" * 64 + wrong = "A" * 63 + "B" + + assert confirm_recovery_key(wrong, canonical) is False + assert confirm_recovery_key("short", canonical) is False + + +def test_lookalike_folding_does_not_collapse_distinct_canonical_keys() -> None: + canonical = "0" + ("A" * 63) + distinct = "1" + ("A" * 63) + + assert confirm_recovery_key("O" + ("A" * 63), canonical) is True + assert confirm_recovery_key("O" + ("A" * 63), distinct) is False diff --git a/tests/test_backup_repo.py b/tests/test_backup_repo.py new file mode 100644 index 000000000..266f39f86 --- /dev/null +++ b/tests/test_backup_repo.py @@ -0,0 +1,248 @@ +# SPDX-License-Identifier: AGPL-3.0-only +# Copyright (c) 2026 sol pbc + +from __future__ import annotations + +import shutil +from pathlib import Path +from typing import Any + +import pytest + +from solstone.think.backup import repo +from solstone.think.backup.destination import ( + Destination, + DestinationStatus, + assemble_backend_env, + validate_destination, +) +from solstone.think.backup.runner import ResticResult, run_restic + +RESTIC_BIN = shutil.which("restic") + + +def _destination(repository: str = "s3:safe-bucket/path") -> Destination: + return Destination( + repository=repository, + backend="s3", + credentials={ + "access_key_id": "access-key", + "secret_access_key": "secret-key", + }, + ) + + +def _status(reason_code: str) -> DestinationStatus: + if reason_code == "repo_exists": + return DestinationStatus(True, True, "repo_exists", "exists") + if reason_code == "repo_missing": + return DestinationStatus(True, False, "repo_missing", "missing") + if reason_code == "auth_failed": + return DestinationStatus(True, True, "auth_failed", "bad password") + if reason_code == "locked": + return DestinationStatus(True, True, "locked", "repository is locked") + return DestinationStatus(False, False, reason_code, "could not reach") + + +def test_init_repository_missing_repo_initializes_adds_and_verifies( + monkeypatch: pytest.MonkeyPatch, +) -> None: + statuses = iter([_status("repo_missing"), _status("repo_exists")]) + calls: list[tuple[str, Any]] = [] + + def fake_validate(destination, password, **kwargs): + calls.append(("validate", password)) + return next(statuses) + + def fake_run_restic(args: list[str], **kwargs: Any) -> ResticResult: + calls.append(("run", tuple(args), kwargs.get("pass_fds", ()))) + return ResticResult(0, "", "", None, ("restic", *args)) + + monkeypatch.setattr(repo, "validate_destination", fake_validate) + monkeypatch.setattr(repo, "run_restic", fake_run_restic) + + repo.init_repository( + _destination(), + daily_key="daily", + recovery_key="A" * 64, + restic_path=Path("/usr/bin/restic"), + ) + + assert calls[0] == ("validate", "daily") + assert calls[1] == ("run", ("init",), ()) + assert calls[2][0] == "run" + assert calls[2][1][:3] == ("key", "add", "--new-password-file") + assert calls[2][2] + assert calls[3] == ("validate", "A" * 64) + + +def test_init_repository_existing_repo_with_recovery_key_noops( + monkeypatch: pytest.MonkeyPatch, +) -> None: + statuses = iter([_status("repo_exists"), _status("repo_exists")]) + calls: list[tuple[str, Any]] = [] + + def fake_validate(destination, password, **kwargs): + calls.append(("validate", password)) + return next(statuses) + + def fail_run_restic(args: list[str], **kwargs: Any) -> ResticResult: + raise AssertionError("restic should not be called") + + monkeypatch.setattr(repo, "validate_destination", fake_validate) + monkeypatch.setattr(repo, "run_restic", fail_run_restic) + + repo.init_repository( + _destination(), + daily_key="daily", + recovery_key="recovery", + restic_path=Path("/usr/bin/restic"), + ) + + assert calls == [("validate", "daily"), ("validate", "recovery")] + + +def test_init_repository_existing_repo_adds_missing_recovery_key( + monkeypatch: pytest.MonkeyPatch, +) -> None: + statuses = iter( + [_status("repo_exists"), _status("auth_failed"), _status("repo_exists")] + ) + calls: list[tuple[str, Any]] = [] + + def fake_validate(destination, password, **kwargs): + calls.append(("validate", password)) + return next(statuses) + + def fake_run_restic(args: list[str], **kwargs: Any) -> ResticResult: + calls.append(("run", tuple(args))) + return ResticResult(0, "", "", None, ("restic", *args)) + + monkeypatch.setattr(repo, "validate_destination", fake_validate) + monkeypatch.setattr(repo, "run_restic", fake_run_restic) + + repo.init_repository( + _destination(), + daily_key="daily", + recovery_key="recovery", + restic_path=Path("/usr/bin/restic"), + ) + + assert calls[0] == ("validate", "daily") + assert calls[1] == ("validate", "recovery") + assert calls[2][0] == "run" + assert calls[2][1][:3] == ("key", "add", "--new-password-file") + assert calls[3] == ("validate", "recovery") + + +def test_init_repository_daily_auth_failed_raises_conflict( + monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.setattr( + repo, + "validate_destination", + lambda *args, **kwargs: _status("auth_failed"), + ) + + def fail_run_restic(args: list[str], **kwargs: Any) -> ResticResult: + raise AssertionError("restic should not be called") + + monkeypatch.setattr(repo, "run_restic", fail_run_restic) + + with pytest.raises(RuntimeError, match="configured daily key"): + repo.init_repository( + _destination(), + daily_key="daily", + recovery_key="recovery", + restic_path=Path("/usr/bin/restic"), + ) + + +@pytest.mark.parametrize("reason_code", ["locked", "timeout", "unreachable"]) +def test_init_repository_unavailable_status_raises_without_restic( + monkeypatch: pytest.MonkeyPatch, + reason_code: str, +) -> None: + monkeypatch.setattr( + repo, + "validate_destination", + lambda *args, **kwargs: _status(reason_code), + ) + + def fail_run_restic(args: list[str], **kwargs: Any) -> ResticResult: + raise AssertionError("restic should not be called") + + monkeypatch.setattr(repo, "run_restic", fail_run_restic) + + with pytest.raises(RuntimeError): + repo.init_repository( + _destination(), + daily_key="daily", + recovery_key="recovery", + restic_path=Path("/usr/bin/restic"), + ) + + +def test_add_recovery_key_raises_sanitized_error( + monkeypatch: pytest.MonkeyPatch, +) -> None: + def fake_run_restic(args: list[str], **kwargs: Any) -> ResticResult: + return ResticResult( + 42, + "", + "presigned-url-signature", + None, + ("restic", *args), + ) + + monkeypatch.setattr(repo, "run_restic", fake_run_restic) + + with pytest.raises(RuntimeError) as exc_info: + repo._add_recovery_key( + _destination(), + daily_key="daily", + recovery_key="recovery", + restic_path=Path("/usr/bin/restic"), + ) + + assert str(exc_info.value) == "restic key add failed with returncode 42" + + +@pytest.mark.skipif(RESTIC_BIN is None, reason="restic is not installed") +def test_init_and_add_recovery_key_local_repository_integration(tmp_path: Path) -> None: + restic_path = Path(RESTIC_BIN or "") + destination = _destination(f"local:{tmp_path / 'repo'}") + + init_result = run_restic( + ["init"], + repository=destination.repository, + password="daily-password", + restic_path=restic_path, + backend_env=assemble_backend_env(destination), + timeout=15, + ) + assert init_result.returncode == 0, init_result.stderr + + repo._add_recovery_key( + destination, + daily_key="daily-password", + recovery_key="recovery-password", + restic_path=restic_path, + timeout=15, + ) + + daily_status = validate_destination( + destination, + "daily-password", + restic_path=restic_path, + timeout=15, + ) + recovery_status = validate_destination( + destination, + "recovery-password", + restic_path=restic_path, + timeout=15, + ) + + assert daily_status.reason_code == "repo_exists" + assert recovery_status.reason_code == "repo_exists" diff --git a/tests/test_backup_runner.py b/tests/test_backup_runner.py index 8a3cf3505..25642cea5 100644 --- a/tests/test_backup_runner.py +++ b/tests/test_backup_runner.py @@ -22,6 +22,7 @@ def test_run_restic_builds_safe_argv_and_minimal_env( def fake_run(argv: list[str], **kwargs: Any) -> subprocess.CompletedProcess[str]: captured["argv"] = argv captured["env"] = kwargs["env"] + captured["pass_fds"] = kwargs["pass_fds"] return subprocess.CompletedProcess(argv, 0, stdout="{}", stderr="") monkeypatch.setenv("PATH", "/bin") @@ -62,6 +63,36 @@ def test_run_restic_builds_safe_argv_and_minimal_env( "AWS_ACCESS_KEY_ID": "access-key", "AWS_SECRET_ACCESS_KEY": "backend-secret", } + assert captured["pass_fds"] == () + + +def test_run_restic_threads_pass_fds(monkeypatch: pytest.MonkeyPatch): + captured: dict[str, Any] = {} + + def fake_run(argv: list[str], **kwargs: Any) -> subprocess.CompletedProcess[str]: + captured["argv"] = argv + captured["pass_fds"] = kwargs["pass_fds"] + return subprocess.CompletedProcess(argv, 0, stdout="", stderr="") + + monkeypatch.setattr(runner.subprocess, "run", fake_run) + + result = runner.run_restic( + ["key", "add", "--new-password-file", "/dev/fd/17"], + repository="s3:safe-bucket/path", + password="repo-password", + restic_path=Path("/usr/bin/restic"), + pass_fds=(17,), + ) + + assert result.returncode == 0 + assert captured["argv"] == [ + "/usr/bin/restic", + "key", + "add", + "--new-password-file", + "/dev/fd/17", + ] + assert captured["pass_fds"] == (17,) def test_run_restic_scrubs_success_output_and_json( diff --git a/tests/test_backup_state.py b/tests/test_backup_state.py new file mode 100644 index 000000000..0e10c44c6 --- /dev/null +++ b/tests/test_backup_state.py @@ -0,0 +1,204 @@ +# SPDX-License-Identifier: AGPL-3.0-only +# Copyright (c) 2026 sol pbc + +from __future__ import annotations + +import json +import stat +from contextlib import contextmanager +from pathlib import Path + +import pytest + +from solstone.think.backup import state +from solstone.think.backup.destination import Destination + + +def _config_path(journal: Path) -> Path: + return journal / "config" / "journal.json" + + +def _write_config(journal: Path, payload: dict) -> None: + config_path = _config_path(journal) + config_path.parent.mkdir(parents=True, exist_ok=True) + config_path.write_text(json.dumps(payload), encoding="utf-8") + + +def _read_config(journal: Path) -> dict: + return json.loads(_config_path(journal).read_text(encoding="utf-8")) + + +def test_missing_backup_section_defaults( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.setenv("SOLSTONE_JOURNAL", str(tmp_path)) + _write_config(tmp_path, {"identity": {"name": "Test"}}) + + config = state.get_backup_config() + + assert config == state.BACKUP_DEFAULTS + assert state.get_destination() is None + assert state.get_keys() is None + + +def test_partial_backup_section_gets_per_field_defaults( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.setenv("SOLSTONE_JOURNAL", str(tmp_path)) + _write_config(tmp_path, {"backup": {"enabled": True}}) + + config = state.get_backup_config() + + assert config["enabled"] is True + assert config["mode"] == "byo" + assert config["destination"] == state.BACKUP_DEFAULTS["destination"] + assert config["retention"] == state.BACKUP_DEFAULTS["retention"] + assert config["schedule"] == state.BACKUP_DEFAULTS["schedule"] + assert config["last_backup"] == state.BACKUP_DEFAULTS["last_backup"] + + +def test_generate_and_store_keys_get_or_create( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.setenv("SOLSTONE_JOURNAL", str(tmp_path)) + _write_config(tmp_path, {}) + monkeypatch.setattr(state, "generate_daily_key", lambda: "generated-daily") + monkeypatch.setattr(state, "generate_recovery_key", lambda: "A" * 64) + + first = state.generate_and_store_keys() + + assert first.daily_key == "generated-daily" + assert first.recovery_key == "A" * 64 + assert _read_config(tmp_path)["backup"]["daily_key"] == "generated-daily" + + monkeypatch.setattr(state, "generate_daily_key", lambda: "new-daily") + monkeypatch.setattr(state, "generate_recovery_key", lambda: "B" * 64) + + second = state.generate_and_store_keys() + + assert second == first + assert _read_config(tmp_path)["backup"]["recovery_key"] == "A" * 64 + + +def test_generate_and_store_keys_preserves_hand_set_keys( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.setenv("SOLSTONE_JOURNAL", str(tmp_path)) + _write_config( + tmp_path, + { + "backup": { + "daily_key": "manual-daily", + "recovery_key": "B" * 64, + } + }, + ) + + def fail_generate() -> str: + raise AssertionError("existing keys must not be regenerated") + + monkeypatch.setattr(state, "generate_daily_key", fail_generate) + monkeypatch.setattr(state, "generate_recovery_key", fail_generate) + + keys = state.generate_and_store_keys() + + assert keys.daily_key == "manual-daily" + assert keys.recovery_key == "B" * 64 + + +def test_set_destination_writes_private_config_mode( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.setenv("SOLSTONE_JOURNAL", str(tmp_path)) + _write_config(tmp_path, {}) + + state.set_destination( + Destination( + repository="s3:safe-bucket/path", + backend="s3", + credentials={ + "access_key_id": "access-key", + "secret_access_key": "secret-key", + }, + ) + ) + + assert stat.S_IMODE(_config_path(tmp_path).stat().st_mode) == 0o600 + + +def test_setters_round_trip_under_config_lock( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.setenv("SOLSTONE_JOURNAL", str(tmp_path)) + _write_config(tmp_path, {}) + entries = 0 + + @contextmanager + def fake_lock(): + nonlocal entries + entries += 1 + yield + + monkeypatch.setattr(state, "hold_config_lock", fake_lock) + destination = Destination( + repository="b2:bucket:path", + backend="b2", + credentials={ + "account_id": "account-id", + "account_key": "account-key", + }, + ) + + state.set_destination(destination) + state.set_recovery_key_confirmed() + + assert entries == 2 + assert state.get_destination() == destination + assert _read_config(tmp_path)["backup"]["confirmed_recovery_key"] is True + + +def test_status_view_redacts_all_secrets( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.setenv("SOLSTONE_JOURNAL", str(tmp_path)) + _write_config( + tmp_path, + { + "backup": { + "enabled": True, + "mode": "byo", + "destination": { + "repository": "s3:safe-bucket/path", + "backend": "s3", + "credentials": { + "access_key_id": "access-key", + "secret_access_key": "secret-key", + }, + }, + "daily_key": "daily-secret", + "recovery_key": "C" * 64, + "confirmed_recovery_key": True, + } + }, + ) + + view = state.status_view() + serialized = json.dumps(view) + + for secret in ("daily-secret", "C" * 64, "access-key", "secret-key"): + assert secret not in serialized + assert view["destination"] == { + "repository": "s3:safe-bucket/path", + "backend": "s3", + "credentials_set": True, + } + assert view["daily_key_set"] is True + assert view["recovery_key_set"] is True + assert view["recovery_key_confirmed"] is True