diff --git a/docs/reference/account-migration.md b/docs/reference/account-migration.md index 32ba70e..0b50d2c 100644 --- a/docs/reference/account-migration.md +++ b/docs/reference/account-migration.md @@ -50,27 +50,37 @@ to the full migration-in sequence: ```bash export OLD_PDS="https://jellybaby.us-east.host.bsky.network" +export OLD_AUTH_PDS="$OLD_PDS" +export OLD_LOGIN_PDS="$OLD_AUTH_PDS" export HANDLE="tempestpds.bsky.social" export DID="did:plc:oga6ppys7zwxlheuqmcm7dac" export TEMPEST="https://tempest.desertthunder.dev" export TEMPEST_SERVICE_DID="did:web:tempest.desertthunder.dev" export EMAIL="operator@example.com" -read -s OLD_PASSWORD -read -s TEMPEST_PASSWORD +export OLD_IDENTIFIER="$HANDLE" +read -rs OLD_PASSWORD +export OLD_PASSWORD +read -rs TEMPEST_PASSWORD +export TEMPEST_PASSWORD # Optional, only if the source PDS requires an auth-factor token/code during # createSession. -read -s OLD_AUTH_FACTOR_TOKEN +read -rs OLD_AUTH_FACTOR_TOKEN export OLD_AUTH_FACTOR_TOKEN UV_CACHE_DIR=.sandbox/uv-cache uv run --project scripts tempest ``` +Use `read -rs` or single quotes for passwords. Unquoted shell assignments can +expand characters like `$`, so a password containing `$N` will not be passed +literally. + Run individual steps when resuming or inspecting a failure: ```bash UV_CACHE_DIR=.sandbox/uv-cache uv run --project scripts tempest login-source UV_CACHE_DIR=.sandbox/uv-cache uv run --project scripts tempest service-auth +UV_CACHE_DIR=.sandbox/uv-cache uv run --project scripts tempest source-session-status UV_CACHE_DIR=.sandbox/uv-cache uv run --project scripts tempest export-car UV_CACHE_DIR=.sandbox/uv-cache uv run --project scripts tempest list-source-blobs UV_CACHE_DIR=.sandbox/uv-cache uv run --project scripts tempest download-source-blobs @@ -183,6 +193,31 @@ If `plc-request-token` returns `Bad token scope`, refresh the source session wit the main account password rather than an app password. If the source PDS requires an auth-factor token/code for high-risk account operations, set `OLD_AUTH_FACTOR_TOKEN` and rerun `login-source`, then rerun `plc-request-token`. +If handle login returns `Invalid identifier or password`, set `OLD_IDENTIFIER` +to the account email address and retry `login-source`. + +If the account's repository host rejects main-password login even though the +credentials work in Bluesky, keep `OLD_PDS` pointed at the repository host for +CAR/blob export, keep `OLD_AUTH_PDS` pointed at the old PDS for authenticated +PLC operations, and set `OLD_LOGIN_PDS` to the Bluesky entryway: + +```bash +export OLD_PDS="https://jellybaby.us-east.host.bsky.network" +export OLD_AUTH_PDS="$OLD_PDS" +export OLD_LOGIN_PDS="https://bsky.social" +UV_CACHE_DIR=.sandbox/uv-cache uv run --project scripts tempest login-source +UV_CACHE_DIR=.sandbox/uv-cache uv run --project scripts tempest plc-request-token +``` + +If login succeeds but `plc-request-token` still returns `Bad token scope`, check +that the saved token works for ordinary old-PDS auth: + +```bash +UV_CACHE_DIR=.sandbox/uv-cache uv run --project scripts tempest source-session-status +``` + +If `source-session-status` succeeds while `plc-request-token` fails, the source +session is valid but the old PDS is refusing that session scope for PLC signing. Before submitting, inspect the signed operation. The PDS service endpoint must be Tempest: @@ -254,6 +289,15 @@ with private rotation-key material; optionally configure `TEMPEST_PLC_RECOVERY_K for an operator recovery key. Tempest derives public `did:key` rotation keys from that material and does not reuse repository signing keys as PLC rotation keys. +The vendored lexicon files for this flow live in `priv/lexicons/official`: + +- `com/atproto/identity/requestPlcOperationSignature.json`: no input body; asks + the old PDS to email a PLC operation code. +- `com/atproto/identity/signPlcOperation.json`: takes the emailed `token` plus + `rotationKeys`, `alsoKnownAs`, `verificationMethods`, and `services`. +- `com/atproto/server/createSession.json`: takes `identifier`, `password`, and + optional `authFactorToken`. + ## Verification Local lifecycle coverage: diff --git a/docs/reference/tokens.md b/docs/reference/tokens.md index c395427..b386eea 100644 --- a/docs/reference/tokens.md +++ b/docs/reference/tokens.md @@ -33,22 +33,42 @@ example: ```bash export OLD_PDS="https://jellybaby.us-east.host.bsky.network" +export OLD_AUTH_PDS="$OLD_PDS" +export OLD_LOGIN_PDS="$OLD_AUTH_PDS" export HANDLE="tempestpds.bsky.social" +export OLD_IDENTIFIER="$HANDLE" export TEMPEST="https://tempest.desertthunder.dev" export TEMPEST_SERVICE_DID="did:web:tempest.desertthunder.dev" -read -s OLD_PASSWORD +read -rs OLD_PASSWORD +export OLD_PASSWORD ``` Use the main account password for PLC operation signing. App passwords can be useful for ordinary source-PDS access, but they may produce a session that cannot request a PLC operation signature. +Do not assign passwords with unquoted shell syntax. Characters like `$`, `!`, +backticks, and backslashes can be expanded or interpreted by the shell. Prefer +silent input: + +```bash +unset OLD_PASSWORD +read -rs OLD_PASSWORD +export OLD_PASSWORD +``` + +If assigning directly, use single quotes: + +```bash +export OLD_PASSWORD='literal-password-with-$N' +``` + If the source PDS asks for an auth-factor token/code during login, pass it to the CLI as `OLD_AUTH_FACTOR_TOKEN`: ```bash -read -s OLD_AUTH_FACTOR_TOKEN +read -rs OLD_AUTH_FACTOR_TOKEN export OLD_AUTH_FACTOR_TOKEN ``` @@ -157,7 +177,7 @@ source PDS emails a code instead, keep it out of shell history when possible and export it only for the signing step: ```bash -read -s PLC_TOKEN +read -rs PLC_TOKEN export PLC_TOKEN UV_CACHE_DIR=.sandbox/uv-cache uv run --project scripts tempest plc-sign ``` @@ -173,7 +193,42 @@ For this deployment the value must be `https://tempest.desertthunder.dev`. If `plc-request-token` returns `Bad token scope`, the source session is not authorized for PLC signing. Re-run `login-source` with the main account password -and any required `OLD_AUTH_FACTOR_TOKEN`, then retry `plc-request-token`. +and any required `OLD_AUTH_FACTOR_TOKEN`, then retry `plc-request-token`. If +handle login returns `Invalid identifier or password`, set `OLD_IDENTIFIER` to +the source account email address and retry `login-source`. + +After a successful `login-source`, verify whether the same token works for +ordinary old-PDS auth: + +```bash +UV_CACHE_DIR=.sandbox/uv-cache uv run --project scripts tempest source-session-status +``` + +If that succeeds while `plc-request-token` fails, the saved source session is +valid and the source PDS is specifically refusing PLC signing for that token +scope. + +Local lexicon references: + +- `priv/lexicons/official/com/atproto/identity/requestPlcOperationSignature.json` + defines this as an authenticated no-input procedure that requests an emailed + code. +- `priv/lexicons/official/com/atproto/identity/signPlcOperation.json` defines + the `token` field consumed by signing. +- `priv/lexicons/official/com/atproto/server/createSession.json` defines the + optional `authFactorToken` used during session creation. + +If the repository host rejects main-password login, try a separate source login +host while leaving `OLD_PDS` unchanged for repo and blob export and +`OLD_AUTH_PDS` unchanged for authenticated old-PDS operations: + +```bash +export OLD_PDS="https://jellybaby.us-east.host.bsky.network" +export OLD_AUTH_PDS="$OLD_PDS" +export OLD_LOGIN_PDS="https://bsky.social" +UV_CACHE_DIR=.sandbox/uv-cache uv run --project scripts tempest login-source +UV_CACHE_DIR=.sandbox/uv-cache uv run --project scripts tempest plc-request-token +``` ## Safety Notes diff --git a/lib/tempest_web/router.ex b/lib/tempest_web/router.ex index d51da61..bb4ce48 100644 --- a/lib/tempest_web/router.ex +++ b/lib/tempest_web/router.ex @@ -32,8 +32,8 @@ defmodule TempestWeb.Router do end pipeline :xrpc do - plug :accepts, ["json"] plug :put_xrpc_cors_headers + plug :accepts, ["json"] plug TempestWeb.Plugs.XrpcAuth end @@ -119,6 +119,11 @@ defmodule TempestWeb.Router do conn |> Plug.Conn.put_resp_header("access-control-allow-origin", "*") |> Plug.Conn.put_resp_header("access-control-allow-methods", "GET, POST, OPTIONS") - |> Plug.Conn.put_resp_header("access-control-allow-headers", "authorization, content-type") + |> Plug.Conn.put_resp_header( + "access-control-allow-headers", + "authorization, content-type, dpop, atproto-proxy, atproto-accept-labelers, x-atproto-accept-labelers" + ) + |> Plug.Conn.put_resp_header("access-control-expose-headers", "dpop-nonce") + |> Plug.Conn.put_resp_header("access-control-max-age", "86400") end end diff --git a/scripts/src/tempest_py/main.py b/scripts/src/tempest_py/main.py index 2ad1828..932b752 100644 --- a/scripts/src/tempest_py/main.py +++ b/scripts/src/tempest_py/main.py @@ -36,6 +36,7 @@ class Command(StrEnum): FULL = "full" LOGIN_SOURCE = "login-source" SERVICE_AUTH = "service-auth" + SOURCE_SESSION_STATUS = "source-session-status" EXPORT_CAR = "export-car" LIST_SOURCE_BLOBS = "list-source-blobs" DOWNLOAD_SOURCE_BLOBS = "download-source-blobs" @@ -56,6 +57,8 @@ class Command(StrEnum): class Settings: artifact_dir: Path old_pds: str + old_login_pds: str + old_auth_pds: str tempest: str tempest_service_did: str did: str @@ -100,6 +103,8 @@ def settings_from_env(args: argparse.Namespace) -> Settings: return Settings( artifact_dir=artifact_dir, old_pds=env("OLD_PDS", DEFAULT_OLD_PDS).rstrip("/"), + old_auth_pds=env("OLD_AUTH_PDS", env("OLD_PDS", DEFAULT_OLD_PDS)).rstrip("/"), + old_login_pds=env("OLD_LOGIN_PDS", env("OLD_AUTH_PDS", env("OLD_PDS", DEFAULT_OLD_PDS))).rstrip("/"), tempest=env("TEMPEST", DEFAULT_TEMPEST).rstrip("/"), tempest_service_did=env("TEMPEST_SERVICE_DID", DEFAULT_TEMPEST_SERVICE_DID), did=env("DID", DEFAULT_DID), @@ -226,6 +231,21 @@ def access_from_session(settings: Settings) -> str: return explicit data = read_json(settings.old_session_path) + session_host = data.get("_tempest_old_login_pds") or data.get("_tempest_old_auth_pds") + if isinstance(session_host, str) and session_host != settings.old_login_pds: + raise CliError( + f"{settings.old_session_path} was created for {session_host}, " + f"but OLD_LOGIN_PDS is {settings.old_login_pds}; rerun login-source successfully" + ) + + session_identifier = data.get("_tempest_old_identifier") + current_identifier = env("OLD_IDENTIFIER", settings.handle) + if isinstance(session_identifier, str) and session_identifier != current_identifier: + raise CliError( + f"{settings.old_session_path} was created for identifier {session_identifier}, " + f"but OLD_IDENTIFIER is {current_identifier}; rerun login-source successfully" + ) + token = data.get("accessJwt") if not isinstance(token, str) or not token: raise CliError(f"{settings.old_session_path} does not contain accessJwt") @@ -283,13 +303,25 @@ def plc_operation_token(settings: Settings) -> str: def login_source(settings: Settings) -> None: step("source session") password = require_env(settings, "OLD_PASSWORD", settings.old_password) - url = f"{settings.old_pds}/xrpc/com.atproto.server.createSession" - payload = {"identifier": settings.handle, "password": password} + url = f"{settings.old_login_pds}/xrpc/com.atproto.server.createSession" + identifier = env("OLD_IDENTIFIER", settings.handle) + payload = {"identifier": identifier, "password": password} auth_factor_token = env("OLD_AUTH_FACTOR_TOKEN") if auth_factor_token: payload["authFactorToken"] = auth_factor_token + log( + "login context: " + f"old_login_pds={settings.old_login_pds} " + f"old_auth_pds={settings.old_auth_pds} " + f"identifier={identifier} " + f"has_old_password={bool(password)} " + f"has_old_auth_factor_token={bool(auth_factor_token)}" + ) status, _headers, raw = request("POST", url, json_body=payload) data = expect_json(status, raw, url) + data["_tempest_old_login_pds"] = settings.old_login_pds + data["_tempest_old_auth_pds"] = settings.old_auth_pds + data["_tempest_old_identifier"] = identifier write_json(settings.old_session_path, data) print_json_summary("saved source session", data) log(f"wrote {settings.old_session_path}") @@ -307,6 +339,14 @@ def get_service_auth(settings: Settings) -> None: log(f"wrote {settings.service_auth_path}") +def source_session_status(settings: Settings) -> None: + step("source session status") + url = f"{settings.old_auth_pds}/xrpc/com.atproto.server.getSession" + status, _headers, raw = request("GET", url, headers=bearer(access_from_session(settings))) + data = expect_json(status, raw, url) + print_json_summary("source session is accepted", data) + + def export_car(settings: Settings) -> None: step("export repo CAR") query = urllib.parse.urlencode({"did": settings.did}) @@ -477,10 +517,21 @@ def plc_recommended(settings: Settings) -> None: def plc_request_token(settings: Settings) -> None: step("request PLC operation token from old PDS") - url = f"{settings.old_pds}/xrpc/com.atproto.identity.requestPlcOperationSignature" + url = f"{settings.old_auth_pds}/xrpc/com.atproto.identity.requestPlcOperationSignature" status, _headers, raw = request("POST", url, headers=bearer(access_from_session(settings))) - if status == 400 and settings.old_password: + first_error: dict[str, Any] = {} + if not 200 <= status <= 299: + try: + decoded = json.loads(raw.decode("utf-8")) + first_error = decoded if isinstance(decoded, dict) else {} + except json.JSONDecodeError: + first_error = {} + + first_message = first_error.get("message") + first_error_name = first_error.get("error") + + if status == 400 and settings.old_password and first_message == "password is required": status, _headers, raw = request( "POST", url, @@ -488,6 +539,22 @@ def plc_request_token(settings: Settings) -> None: json_body={"password": settings.old_password}, ) + if not 200 <= status <= 299: + try: + decoded = json.loads(raw.decode("utf-8")) + data = decoded if isinstance(decoded, dict) else {} + except json.JSONDecodeError: + data = first_error + + message = data.get("message") or first_message + error_name = data.get("error") or first_error_name + if message == "Bad token scope": + raise CliError( + "source PDS rejected the fresh session for PLC operation signatures: " + f"{error_name or 'InvalidRequest'}: {message}. " + "The source session itself is valid, but this old-PDS endpoint is refusing its scope." + ) + data = expect_json(status, raw, url) write_json(settings.plc_token_path, data) print_json_summary("saved PLC operation token response", data) @@ -506,7 +573,7 @@ def plc_sign(settings: Settings) -> None: } payload["token"] = plc_operation_token(settings) - url = f"{settings.old_pds}/xrpc/com.atproto.identity.signPlcOperation" + url = f"{settings.old_auth_pds}/xrpc/com.atproto.identity.signPlcOperation" status, _headers, raw = request("POST", url, headers=bearer(access_from_session(settings)), json_body=payload) data = expect_json(status, raw, url) write_json(settings.plc_signed_path, data) @@ -545,6 +612,8 @@ def full(settings: Settings) -> None: started = time.monotonic() log("Tempest migration CLI") log(f"old_pds={settings.old_pds}") + log(f"old_login_pds={settings.old_login_pds}") + log(f"old_auth_pds={settings.old_auth_pds}") log(f"tempest={settings.tempest}") log(f"tempest_service_did={settings.tempest_service_did}") log(f"did={settings.did}") @@ -592,6 +661,8 @@ def run_command(command: Command, settings: Settings) -> None: login_source(settings) case Command.SERVICE_AUTH: get_service_auth(settings) + case Command.SOURCE_SESSION_STATUS: + source_session_status(settings) case Command.EXPORT_CAR: export_car(settings) case Command.LIST_SOURCE_BLOBS: diff --git a/test/tempest_web/controllers/xrpc_controller_test.exs b/test/tempest_web/controllers/xrpc_controller_test.exs index bbd0122..5d5b967 100644 --- a/test/tempest_web/controllers/xrpc_controller_test.exs +++ b/test/tempest_web/controllers/xrpc_controller_test.exs @@ -6,12 +6,38 @@ defmodule TempestWeb.XrpcControllerTest do response = json_response(conn, 200) assert get_resp_header(conn, "content-type") == ["application/json; charset=utf-8"] + assert get_resp_header(conn, "access-control-allow-origin") == ["*"] assert response["availableUserDomains"] == [".localhost"] assert response["inviteCodeRequired"] == false assert response["links"]["privacyPolicy"] == nil assert response["links"]["termsOfService"] == nil end + test "XRPC preflight permits browser ATProto client headers", %{conn: conn} do + conn = + conn + |> put_req_header("origin", "https://pdsmoover.com") + |> put_req_header("access-control-request-method", "GET") + |> put_req_header( + "access-control-request-headers", + "authorization,content-type,dpop,atproto-proxy,atproto-accept-labelers" + ) + |> options(~p"/xrpc/com.atproto.server.describeServer") + + assert response(conn, 204) == "" + assert get_resp_header(conn, "access-control-allow-origin") == ["*"] + assert get_resp_header(conn, "access-control-allow-methods") == ["GET, POST, OPTIONS"] + assert get_resp_header(conn, "access-control-expose-headers") == ["dpop-nonce"] + + [allow_headers] = get_resp_header(conn, "access-control-allow-headers") + assert allow_headers =~ "authorization" + assert allow_headers =~ "content-type" + assert allow_headers =~ "dpop" + assert allow_headers =~ "atproto-proxy" + assert allow_headers =~ "atproto-accept-labelers" + assert allow_headers =~ "x-atproto-accept-labelers" + end + test "unknown XRPC method returns JSON error", %{conn: conn} do conn = get(conn, ~p"/xrpc/com.atproto.unknown.method") response = json_response(conn, 404)