diff --git a/.tangled/workflows/publish-docs.yml b/.tangled/workflows/publish-docs.yml index 68b8b84..e3ecab1 100644 --- a/.tangled/workflows/publish-docs.yml +++ b/.tangled/workflows/publish-docs.yml @@ -1,12 +1,17 @@ when: - - event: push - tag: "v*" + - event: ["push"] + branch: ["main"] + tag: ["v*"] engine: nixery +clone: + depth: 50 + dependencies: nixpkgs: - curl + - git - gnutar - xz @@ -18,5 +23,12 @@ steps: - name: build and publish docs to ATProto command: | + if [ "${TANGLED_REF#refs/tags/}" = "$TANGLED_REF" ]; then + changed="$(git diff-tree --no-commit-id --name-only -r HEAD)" + if ! printf '%s\n' "$changed" | grep -Eq '^(README\.md|CHANGELOG\.md|docs/|devlog/|scripts/publish-docs\.zig)'; then + echo "no docs/devlog changes; skipping ATProto docs publish" + exit 0 + fi + fi /tangled/workspace/.zig/zig build ./zig-out/bin/publish-docs diff --git a/devlog/010-the-network-is-input.md b/devlog/010-the-network-is-input.md index ffa7d0f..eb3a768 100644 --- a/devlog/010-the-network-is-input.md +++ b/devlog/010-the-network-is-input.md @@ -1,9 +1,9 @@ # the network is input -devlog 009 ended with "zat is v0.3.0-alpha. no API changes from this." the next release is different. `v0.3.1` is small in surface area, but it changes how the library treats two pieces of AT Protocol reality: +devlog 009 ended with "zat is v0.3.0-alpha. no API changes from this." the next release is different. `v0.3.1` is small in surface area, but it changes two parts of zat's network behavior: -1. identity strings are not just syntax. resolving them crosses a network boundary. -2. failed XRPC calls are not just failed HTTP. the body is protocol data. +1. resolving [AT Protocol handles](https://atproto.com/specs/handle) and [DIDs](https://atproto.com/specs/did) can require DNS lookups and HTTP fetches. +2. [XRPC](https://atproto.com/specs/xrpc) error responses can carry a JSON error envelope that callers need to inspect. the release is three commits: @@ -11,17 +11,21 @@ the release is three commits: - [`8ba4cc0`](https://tangled.org/zat.dev/zat/commit/8ba4cc0) - add checked xrpc errors and retries - [`8de5f40`](https://tangled.org/zat.dev/zat/commit/8de5f40) - release: v0.3.1 -## identity resolution is a fetch +## identity resolution performs network requests -AT Protocol makes identity resolution look friendly: +AT Protocol account identity uses two related identifiers: [handles and DIDs](https://atproto.com/guides/identity#identifiers). handles are DNS names that resolve to DIDs; DIDs resolve to DID documents with the account's signing key and PDS service endpoint. -- `did:plc:...` goes to `plc.directory` -- `did:web:example.com` goes to `https://example.com/.well-known/did.json` -- `handle.example.com` goes to `https://handle.example.com/.well-known/atproto-did` or `_atproto.handle.example.com` TXT +when zat resolves those identifiers, it may issue these network requests: -that last sentence hides the problem. handles and DIDs are user-controlled strings that can make the library issue HTTP requests. validating the syntax is not enough. `did:web:127.0.0.1` is syntactically ordinary and operationally not something a server should fetch on behalf of an untrusted caller. +- `did:plc:...` resolves through the PLC directory +- `did:web:example.com` resolves through `https://example.com/.well-known/did.json` +- `handle.example.com` resolves through `https://handle.example.com/.well-known/atproto-did` or the `_atproto.handle.example.com` DNS TXT record -the first chunk adds `src/internal/identity/network_safety.zig`. before `did:web` or handle HTTP resolution fetches anything, zat now checks the host and the resolved addresses. the obvious unsafe cases are rejected directly: +handles and DIDs often come from user input, API parameters, repo records, or event streams. when zat resolves one, the library may fetch a URL or ask DNS for an address. syntax validation does not make that safe. `did:web:127.0.0.1` is syntactically ordinary and operationally not something a server should fetch on behalf of an untrusted caller. + +this comes up directly in [`atproto-bench`](https://tangled.org/zzstoatzz.io/atproto-bench). the [full trust-chain verifier](https://tangled.org/zzstoatzz.io/atproto-bench/blob/main/zig/src/verify.zig) accepts a handle or DID, resolves the handle when needed, then resolves the DID document before fetching and verifying the repo. the relay and signature capture harnesses also resolve DIDs from live firehose frames to build signing-key corpora. + +the first chunk adds `src/internal/identity/network_safety.zig`. before `did:web` or handle HTTP resolution fetches anything, zat checks the host and the resolved addresses. obvious unsafe targets are rejected directly: ```zig try std.testing.expectError(error.UnsafeIdentityHost, checkIdentityHost("localhost")); @@ -32,15 +36,15 @@ try std.testing.expectError(error.UnsafeIdentityHost, checkIdentityHost("[::1]") try std.testing.expectError(error.UnsafeIdentityHost, checkIdentityHost("::ffff:127.0.0.1")); ``` -the less obvious case is DNS. `evil.example` can be a public name that resolves to `127.0.0.1`, `10.0.0.5`, `fc00::1`, or a link-local address. so the resolver does a DoH preflight before the HTTP fetch. it asks for `A` and `AAAA`, rejects non-routable answers, and only then dials. +DNS needs a separate check. `evil.example` can be a public name that resolves to `127.0.0.1`, `10.0.0.5`, `fc00::1`, or a link-local address. the resolver now does a DNS-over-HTTPS preflight before the HTTP fetch. it asks for `A` and `AAAA`, rejects non-routable answers, and only then dials. -for `did:web` and handle well-known HTTP, redirects are disabled. redirecting from a safe-looking public URL to a private address is the same bug with one extra step. +for `did:web` and [handle well-known HTTP](https://atproto.com/specs/handle#handle-resolution), redirects are disabled. otherwise, an attacker could provide a safe-looking public URL that redirects zat's server-side fetch to a private address. -## dialing one host while speaking for another +## using the checked address -the DoH preflight created a second constraint: once we have checked an address, the actual HTTP request should use that checked address, not resolve the hostname again underneath `std.http.Client`. +the DNS-over-HTTPS preflight means zat has to preserve the checked address through the HTTP request. once zat has checked an address, it should not hand the hostname back to `std.http.Client` and let it resolve the name a second time. -`HttpTransport` now has an internal `ResolvedConnection` path: +`HttpTransport` now has an internal `ResolvedConnection` mode: ```zig pub const ResolvedConnection = struct { @@ -49,46 +53,46 @@ pub const ResolvedConnection = struct { }; ``` -the transport connects to `dial_host`, but keeps `logical_host` for HTTP/TLS identity. that preserves the thing callers intended to fetch while avoiding a second unchecked resolver hop. it also checks that the request URL still matches the logical host, so the preflight result cannot be accidentally reused for a different URL. +the transport connects to `dial_host`, but keeps `logical_host` for the HTTP `Host` header and TLS server name. that preserves the caller's intended URL while avoiding a second unchecked resolver hop. it also checks that the request URL still matches the logical host, so the preflight result cannot be accidentally reused for a different URL. -this is not meant to be a general proxy API. it is just enough machinery for identity resolution to say: "I already checked where this name points; use that." +this is narrow internal plumbing for identity resolution: "I already checked where this name points; use that address for this request." ## XRPC errors are data -the second chunk came from downstream use. the original XRPC API made this easy: +the second chunk came from downstream use. the original XRPC API made this pattern easy: ```zig var response = try client.query(nsid, params); if (!response.ok()) return error.ApiFailed; ``` -that is fine for a prototype. it is not enough for a client that needs to understand AT Protocol behavior. a non-2xx response can still contain a structured XRPC envelope: +that collapses protocol errors into a boolean. the [XRPC specification](https://atproto.com/specs/xrpc#error-responses) says unsuccessful responses should use a JSON object with an `error` string and optional `message` string: ```json {"error":"RateLimitExceeded","message":"slow down"} ``` -throwing that away means callers lose the difference between `InvalidRequest`, `ExpiredToken`, `RateLimitExceeded`, and an arbitrary 500. it also makes retry behavior hard to centralize because the transport sees the status and headers, while application code sees only a boolean. +discarding that body loses the difference between `InvalidRequest`, `ExpiredToken`, `RateLimitExceeded`, and an arbitrary 500. it also makes retry behavior hard to centralize because the transport sees the status and headers, while application code sees only a boolean. `v0.3.1` adds checked XRPC calls: ```zig -var result = try client.queryChecked(nsid, params, .{}); -defer result.deinit(); +var query_result = client.queryChecked(nsid, params, .{}) catch |err| { + log.err("getAuthorFeed API error for {s}: {}", .{ actor_did, err }); + return error.ApiFailed; +}; +defer query_result.deinit(); -switch (result) { - .ok => |response| { - // parse success body - _ = response; - }, +const response = switch (query_result) { + .ok => |response| response, .err => |xrpc_error| { - // status, error_name, message, body, rate_limit - _ = xrpc_error; + logXrpcError("getAuthorFeed", actor_did, xrpc_error); + return error.ApiFailed; }, -} +}; ``` -the old `query` and `procedure` calls stay. the checked calls are additive, and the return type forces the caller to decide what to do with protocol errors. +that is the pattern now used in [`music-atmosphere-feed`](https://tangled.org/zzstoatzz.io/music-atmosphere-feed/blob/main/src/bsky/api.zig): public functions still return the application's `ApiFailed` error, but logs keep the XRPC status, error name, and message for AppView calls like `app.bsky.graph.getFollows` and `app.bsky.feed.getAuthorFeed`. the old `query` and `procedure` calls stay. the checked calls are additive, and the return type forces the caller to decide what to do with protocol errors. ## retries belong with the client @@ -103,7 +107,9 @@ the same change adds `XrpcClient.RetryPolicy`. the default is conservative: retr those fields are present on both successful responses and `XrpcError`. that matters because a caller might need to surface the error immediately but still update local rate-limit state. -## proof belongs downstream +the local smoke in [`atproto-bench`](https://tangled.org/zzstoatzz.io/atproto-bench/blob/main/zig/src/xrpc_checked.zig) uses this API directly. it runs a fixture server, asks `queryChecked` to retry a `429`, then verifies that a structured `400` carries `InvalidRequest`, `message`, and rate-limit headers through `XrpcError`. + +## smoke tests belong downstream we did not put the smoke harness in zat. zat has unit tests for the pieces: @@ -114,15 +120,15 @@ we did not put the smoke harness in zat. zat has unit tests for the pieces: - XRPC error-envelope parsing - deterministic retry delay behavior -the end-to-end smoke went into [`atproto-bench`](https://tangled.org/zzstoatzz.io/atproto-bench), where this kind of protocol harness belongs. the harness runs a local HTTP fixture, makes `queryChecked` hit a `429`, verifies the retry succeeds, then verifies a structured `400` comes back as `XrpcError` with rate-limit headers intact. +the end-to-end smoke went into [`atproto-bench`](https://tangled.org/zzstoatzz.io/atproto-bench). that repository already holds protocol harnesses and benchmark fixtures, so it is the right place to exercise behavior that spans zat plus a local HTTP server. -then [`music-atmosphere-feed`](https://tangled.org/zzstoatzz.io/music-atmosphere-feed) adopted `queryChecked` for its public AppView calls. that is the useful downstream shape: application code still returns its own `ApiFailed`, but it logs the actual XRPC status, error name, and message instead of flattening everything to "not ok." +then [`music-atmosphere-feed`](https://tangled.org/zzstoatzz.io/music-atmosphere-feed/blob/main/src/bsky/api.zig) adopted `queryChecked` for its public AppView calls. application code keeps returning its own errors, while logs preserve the protocol error details. ## the release -`v0.3.1` is a patch release because the existing API remains available. the new calls are additive, and the identity hardening is a fix to behavior that should not have been allowed by default. +`v0.3.1` is a patch release because the existing API remains available. the new calls are additive, and the identity hardening changes the default from "fetch any syntactically valid identity target" to "reject private-network identity targets." -there is one practical compatibility note: if someone was intentionally resolving `did:web` or handles to private infrastructure through zat's identity resolvers, that now fails with `error.UnsafeIdentityHost`. that is the correct default for a public AT Protocol library. private-network fetching needs an explicit escape hatch, not accidental behavior. +there is one practical compatibility note: if someone was intentionally resolving `did:web` or handles to private infrastructure through zat's identity resolvers, that now fails with `error.UnsafeIdentityHost`. for code that resolves identifiers from untrusted public inputs, blocking private-network targets is the safer default. private-network identity resolution is still a legitimate use case, but it should be configured explicitly instead of happening by accident. the local release checks are boring, which is what a patch release should be: