diff --git a/CHANGELOG.md b/CHANGELOG.md --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,12 @@ # changelog +## 0.2.4 + +- **feat**: configurable CAR size limits — `max_size` and `max_blocks` options in `readWithOptions` for large repo verification +- **feat**: export `jwt` module (not just `Jwt` type) for direct access to `verifySecp256k1`/`verifyP256` +- **docs**: devlog 005 — three-way trust chain verification (zig vs Go vs Rust) +- **docs**: README rewrite — added CBOR, CAR, MST, firehose, jetstream, signing, repo verification + ## 0.2.3 - **docs**: devlog 004 — the sig-verify saga (k256 5×52-bit field, Fermat scalar inversion, three-way bench with rsky) diff --git a/README.md b/README.md --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@
this readme is an ATProto record -→ [view in zat.dev's repository](https://at-me.zzstoatzz.io/view?handle=zat.dev) +> [view in zat.dev's repository](https://at-me.zzstoatzz.io/view?handle=zat.dev) zat publishes these docs as [`site.standard.document`](https://standard.site) records, signed by its DID. @@ -49,35 +49,176 @@
-did resolution - resolve did:plc and did:web to documents +identity resolution - resolve handles and DIDs to documents ```zig -var resolver = zat.DidResolver.init(allocator); -defer resolver.deinit(); +// handle → DID +var handle_resolver = zat.HandleResolver.init(allocator); +defer handle_resolver.deinit(); +const did = try handle_resolver.resolve(zat.Handle.parse("bsky.app").?); +defer allocator.free(did); -const did = zat.Did.parse("did:plc:z72i7hdynmk6r22z27h6tvur").?; -var doc = try resolver.resolve(did); +// DID → document +var did_resolver = zat.DidResolver.init(allocator); +defer did_resolver.deinit(); +var doc = try did_resolver.resolve(zat.Did.parse("did:plc:z72i7hdynmk6r22z27h6tvur").?); defer doc.deinit(); -const handle = doc.handle(); // "bsky.app" -const pds = doc.pdsEndpoint(); // "https://..." -const key = doc.signingKey(); // verification method +const pds = doc.pdsEndpoint(); // "https://..." +const key = doc.signingKey(); // verification method ``` + +supports did:plc (via plc.directory) and did:web. handle resolution via HTTP well-known and DNS TXT.
-handle resolution - resolve handles to DIDs via HTTP well-known +CBOR codec - DAG-CBOR encoding and decoding ```zig -var resolver = zat.HandleResolver.init(allocator); -defer resolver.deinit(); +// decode +const decoded = try zat.cbor.decode(allocator, bytes); +defer decoded.deinit(); -const handle = zat.Handle.parse("bsky.app").?; -const did = try resolver.resolve(handle); -defer allocator.free(did); -// did = "did:plc:z72i7hdynmk6r22z27h6tvur" +// navigate values +const text = decoded.value.getStr("text"); +const cid = decoded.value.getCid("data"); + +// encode (deterministic key ordering) +const encoded = try zat.cbor.encodeAlloc(allocator, value); +defer allocator.free(encoded); ``` + +full DAG-CBOR support: maps, arrays, byte strings, text strings, integers, floats, booleans, null, CID tags (tag 42). deterministic encoding with sorted keys for signature verification. + +
+ +
+CAR codec - Content Addressable aRchive parsing with CID verification + +```zig +// parse with SHA-256 CID verification (default) +const parsed = try zat.car.read(allocator, car_bytes); +defer parsed.deinit(); + +const root_cid = parsed.roots[0]; +for (parsed.blocks.items) |block| { + // block.cid_raw, block.data +} + +// skip verification for trusted local data +const fast = try zat.car.readWithOptions(allocator, car_bytes, .{ + .verify_block_hashes = false, +}); +``` + +enforces size limits (configurable `max_size`, `max_blocks`) matching indigo's production defaults. + +
+ +
+MST - Merkle Search Tree + +```zig +var tree = zat.mst.Mst.init(allocator); +defer tree.deinit(); + +try tree.put(allocator, "app.bsky.feed.post/abc123", value_cid); +const found = tree.get("app.bsky.feed.post/abc123"); +try tree.delete(allocator, "app.bsky.feed.post/abc123"); + +// compute root CID (serialize → hash → CID) +const root = try tree.rootCid(allocator); +``` + +the core data structure of an atproto repo. key layer derived from leading zero bits of SHA-256(key), nodes serialized with prefix compression. + +
+ +
+crypto - signing, verification, key encoding + +```zig +// JWT verification +var token = try zat.Jwt.parse(allocator, token_string); +defer token.deinit(); +try token.verify(public_key_multibase); + +// ECDSA signature verification (P-256 and secp256k1) +try zat.jwt.verifySecp256k1(hash, signature, public_key); +try zat.jwt.verifyP256(hash, signature, public_key); + +// multibase/multicodec key parsing +const key_bytes = try zat.multibase.decode(allocator, "zQ3sh..."); +defer allocator.free(key_bytes); +const parsed = try zat.multicodec.parsePublicKey(key_bytes); +// parsed.key_type: .secp256k1 or .p256 +// parsed.raw: 33-byte compressed public key +``` + +ES256 (P-256) and ES256K (secp256k1) with low-S normalization. RFC 6979 deterministic signing. `did:key` construction and multibase encoding. + +
+ +
+repo verification - full AT Protocol trust chain + +```zig +const result = try zat.verifyRepo(allocator, "pfrazee.com"); +defer result.deinit(); + +// result.did, result.signing_key, result.pds_endpoint +// result.record_count, result.block_count +// result.commit_verified (signature check passed) +// result.root_cid_match (MST rebuild matches commit) +``` + +given a handle or DID, resolves identity, fetches the repo, parses every CAR block with SHA-256 verification, verifies the commit signature, walks the MST, and rebuilds the tree to verify the root CID. + +
+ +
+firehose client - raw CBOR event stream from relay + +```zig +var client = zat.FirehoseClient.init(allocator, .{}); +defer client.deinit(); + +try client.connect(); +while (try client.next()) |event| { + switch (event.header.type) { + .commit => { + const car_data = try zat.car.read(allocator, event.body.blocks); + // process blocks... + }, + else => {}, + } +} +``` + +connects to `com.atproto.sync.subscribeRepos` via WebSocket. decodes binary CBOR frames into typed events. round-robin host rotation with backoff. + +
+ +
+jetstream client - typed JSON event stream + +```zig +var client = zat.JetstreamClient.init(allocator, .{ + .wanted_collections = &.{"app.bsky.feed.post"}, +}); +defer client.deinit(); + +try client.connect(); +while (try client.next()) |event| { + if (event.commit) |commit| { + const record = commit.record; + // process... + } +} +``` + +connects to jetstream (bluesky's JSON event stream). typed events, automatic reconnection with cursor tracking, round-robin across community relays.
@@ -102,30 +243,6 @@
-sync types - enums for firehose/event stream consumption - -```zig -// use in struct definitions for automatic json parsing: -const RepoOp = struct { - action: zat.CommitAction, // .create, .update, .delete - path: []const u8, - cid: ?[]const u8, -}; - -// then exhaustive switch: -switch (op.action) { - .create, .update => processUpsert(op), - .delete => processDelete(op), -} -``` - -- **CommitAction** - `.create`, `.update`, `.delete` -- **EventKind** - `.commit`, `.sync`, `.identity`, `.account`, `.info` -- **AccountStatus** - `.takendown`, `.suspended`, `.deleted`, `.deactivated`, `.desynchronized`, `.throttled` - -
- -
json helpers - navigate nested json without verbose if-chains ```zig @@ -146,42 +263,17 @@
-
-jwt verification - verify service auth tokens +## benchmarks -```zig -var jwt = try zat.Jwt.parse(allocator, token_string); -defer jwt.deinit(); +zat is benchmarked against Go (indigo), Rust (rsky), and Python (atproto) in [atproto-bench](https://tangled.sh/@zzstoatzz.io/atproto-bench): -// check claims -if (jwt.isExpired()) return error.TokenExpired; -if (!std.mem.eql(u8, jwt.payload.aud, expected_audience)) return error.InvalidAudience; - -// verify signature against issuer's public key (from DID document) -try jwt.verify(public_key_multibase); -``` - -supports ES256 (P-256) and ES256K (secp256k1) signing algorithms. - -
- -
-multibase decoding - decode public keys from DID documents - -```zig -const key_bytes = try zat.multibase.decode(allocator, "zQ3sh..."); -defer allocator.free(key_bytes); - -const parsed = try zat.multicodec.parsePublicKey(key_bytes); -// parsed.key_type: .secp256k1 or .p256 -// parsed.raw: 33-byte compressed public key -``` - -
+- **decode**: 290k frames/sec (zig) vs 39k (rust) vs 15k (go) — with CID hash verification +- **sig-verify**: 15k–19k verifies/sec across all three — ECDSA is table stakes +- **trust chain**: full repo verification in ~300ms compute (zig) vs ~410ms (go) vs ~422ms (rust) ## specs -validation follows [atproto.com/specs](https://atproto.com/specs/atp). +validation follows [atproto.com/specs](https://atproto.com/specs/atp). passes the [atproto interop test suite](https://github.com/bluesky-social/atproto-interop-tests) (syntax, crypto, MST vectors). ## versioning @@ -197,4 +289,4 @@ --- -[roadmap](docs/roadmap.md) · [changelog](CHANGELOG.md) +[devlog](devlog/) · [changelog](CHANGELOG.md) diff --git a/devlog/005-three-way-verify.md b/devlog/005-three-way-verify.md --- a/devlog/005-three-way-verify.md +++ b/devlog/005-three-way-verify.md @@ -38,6 +38,8 @@ _pfrazee.com — 192,144 records, 243,470 blocks, 70.6 MB CAR, macOS arm64 (M3 Max)_ +trust chain compute breakdown + | SDK | CAR parse | sig verify | MST walk | MST rebuild | compute total | |-----|----------:|----------:|---------:|------------:|-------------:| | zig (zat) | 81.6ms | 0.6ms | 45.5ms | 172.6ms | **300.4ms** | diff --git a/docs/roadmap.md b/docs/roadmap.md --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -7,20 +7,22 @@ **initial scope** - string primitives with parsing and validation. the philosophy: primitives not frameworks, layered design, zig idioms, minimal scope. **what grew from usage:** -- DID resolution was originally "out of scope" - real projects needed it, so `DidResolver` and `DidDocument` got added -- XRPC client and JSON helpers - same story +- DID/handle resolution — real projects needed it, so `DidResolver`, `DidDocument`, `HandleResolver` got added +- XRPC client and JSON helpers — same story - JWT verification for service auth -- handle resolution via HTTP well-known -- handle resolution via DNS-over-HTTP (community contribution) -- sync types for firehose consumption (`CommitAction`, `EventKind`, `AccountStatus`) +- jetstream client — typed JSON event stream with reconnection (0.1.3) +- firehose client — raw CBOR event stream, DAG-CBOR codec, CAR codec, CID creation (0.1.4) +- MST, ECDSA signing, `did:key` construction, multibase encoding (0.1.9) +- full repo verification — end-to-end trust chain from handle to MST root CID match (0.2.0) +- CID hash verification in CAR parser (0.2.1), size limits (0.2.2) this pattern - start minimal, expand based on real pain - continues. ## now -use zat in real projects. let usage drive what's next. +the library covers the full AT Protocol verification pipeline: identity resolution, repo parsing, signature verification, and MST validation. benchmarked against Go (indigo) and Rust (rsky) in [atproto-bench](https://tangled.sh/@zzstoatzz.io/atproto-bench). -the primitives are reasonably complete. what's missing will show up when people build things. until then, no speculative features. +what's missing will show up when people build things. until then, no speculative features. ## maybe later diff --git a/devlog/img/verify-compute.svg b/devlog/img/verify-compute.svg new file mode 100644 --- /dev/null +++ b/devlog/img/verify-compute.svg @@ -0,0 +1,47 @@ + + +AT Protocol trust chain — compute +192,144 records + + + + + + +zig (zat) + +CAR parse + + +MST walk + +MST rebuild +300ms +go (indigo) + +CAR parse + + +410ms +rust (RustCrypto) + +CAR parse + + +MST walk +422ms +0 +84ms +169ms +253ms +338ms +422ms + +CAR parse + +sig verify + +MST walk + +MST rebuild + \ No newline at end of file diff --git a/devlog/img/verify-total.svg b/devlog/img/verify-total.svg new file mode 100644 --- /dev/null +++ b/devlog/img/verify-total.svg @@ -0,0 +1,59 @@ + + +AT Protocol trust chain — total (network + compute) +192,144 records + + + + + + +zig (zat) + + + +fetch + + + + +10.1s +go (indigo) + + + +fetch + + + +21.2s +rust (RustCrypto) + + + +fetch + + + +9.1s +0 +4.2s +8.5s +12.7s +17.0s +21.2s + +handle + +DID doc + +fetch + +CAR parse + +sig verify + +MST walk + +MST rebuild + \ No newline at end of file