From 042fabda0e3ce0ca97fa55e44e3e734d9f857bbd Mon Sep 17 00:00:00 2001 From: dawn <90008@gaze.systems> Date: Sat, 14 Feb 2026 04:29:52 +0300 Subject: [PATCH] [docs] update agent instructions with latest architecture and style guide --- AGENTS.md | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index e528067..a6ebe7b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -35,9 +35,10 @@ Key design goals: Hydrant consists of several components: - **[`hydrant::ingest::firehose`]**: Connects to an upstream Firehose (Relay) and filters events. It manages the transition between discovery and synchronization. -- **[`hydrant::ingest::worker`]**: Processes buffered Firehose messages concurrently. Verifies signatures, updates repository state, detects gaps for backfill, and persists records. +- **[`hydrant::ingest::worker`]**: Processes buffered Firehose messages concurrently using sharded workers. Verifies signatures, updates repository state (handling account status events like deactivations), detects gaps for backfill, and persists records. - **[`hydrant::crawler`]**: Periodically enumerates the network via `com.atproto.sync.listRepos` to discover new repositories when in full-network mode. -- **[`hydrant::backfill`]**: A dedicated worker that fetches full repository CAR files from PDS instances when a new repo is detected. +- **[`hydrant::resolver`]**: Manages DID resolution and key lookups. Supports multiple PLC directory sources with failover and caching. +- **[`hydrant::backfill`]**: A dedicated worker that fetches full repository CAR files. Uses LIFO prioritization and adaptive concurrency to manage backfill load efficiently. - **[`hydrant::api`]**: An Axum-based XRPC server implementing repository read methods (`getRecord`, `listRecords`) and system stats. It also provides a event stream API via WebSockets. - **Persistence worker** (in `src/main.rs`): Manages periodic background flushes of the LSM-tree and cursor state. @@ -48,11 +49,15 @@ To minimize latency in `apply_commit` and the backfill worker, events are stored ## General conventions ### Correctness over convenience -- Model the full error space—no shortcuts or simplified error handling. - Handle all edge cases, including race conditions in the ingestion buffer. - Use the type system to encode correctness constraints. - Prefer compile-time guarantees over runtime checks where possible. +### Error handling +- **Typed Errors**: Define custom error enums (e.g. `ResolverError`, `IngestError`) when callers need to handle specific cases (like rate limits or retries). +- **Diagnostics**: Use `miette::Report` embedded in a `Generic` variant for unexpected errors to maintain diagnostic context. +- **Type Preservation**: Avoid erasing error types with `.into_diagnostic()` in valid code paths; only use it at the top-level application boundary or when the error is truly unrecoverable and needs no special handling. + ### Production-grade engineering - Use `miette` for diagnostic-driven error reporting. - Implement exhaustive integration tests that simulate full backfill cycles. @@ -69,26 +74,30 @@ To minimize latency in `apply_commit` and the backfill worker, events are stored Hydrant uses multiple `fjall` keyspaces: - `repos`: Maps `{DID}` -> `RepoState` (MessagePack). -- `records`: Maps `{DID}|{Collection}|{RKey}` -> `{CID}` (String). +- `records`: Partitioned by collection. Maps `{DID}|{RKey}` -> `{CID}` (Binary). - `blocks`: Maps `{CID}` -> `Block Data` (Raw CBOR). - `events`: Maps `{ID}` (u64) -> `StoredEvent` (MessagePack). This is the source for the JSON stream API. - `cursors`: Maps `firehose_cursor` or `crawler_cursor` -> `Value` (u64/i64 BE Bytes). -- `pending`: Index of DIDs awaiting backfill. +- `pending`: Queue of `{Timestamp}|{DID}` -> `Empty` (Backfill queue). - `resync`: Maps `{DID}` -> `ResyncState` (MessagePack) for retry logic/tombstones. +- `resync_buffer`: Maps `{DID}|{Rev}` -> `Commit` (MessagePack). Used to buffer live events during backfill. - `counts`: Maps `k|{NAME}` or `r|{DID}|{COL}` -> `Count` (u64 BE Bytes). ## Safe commands ### Testing - `nu tests/repo_sync_integrity.nu` - Runs the full integration test suite using Nushell. This builds the binary, starts a temporary instance, performs a backfill against a real PDS, and verifies record integrity. +- `nu tests/verify_crawler.nu` - Verifies full-network crawler functionality using a mock relay. - `nu tests/stream_test.nu` - Tests WebSocket streaming functionality. Verifies both live event streaming during backfill and historical replay with cursor. - `nu tests/authenticated_stream_test.nu` - Tests authenticated event streaming. Verifies that create, update, and delete actions on a real account are correctly streamed by Hydrant in the correct order. Requires `TEST_REPO` and `TEST_PASSWORD` in `.env`. - `nu tests/debug_endpoints.nu` - Tests debug/introspection endpoints (`/debug/iter`, `/debug/get`) and verifies DB content and serialization. ## Rust code style -- Always try to use variable substitution in `format!` like macros (eg. logging macros like `info!`, `debug!`) like so: `format!("error: {err}")`. +- Prefer variable substitution in `format!` like macros (eg. logging macros like `info!`, `debug!`) like so: `format!("error: {err}")`. - Prefer using let-guard (eg. `let Some(val) = res else { continue; }`) over nested ifs where it makes sense (eg. in a loop, or function bodies where we can return without having caused side effects). +- Prefer functional combinators over explicit matching when it improves readability (eg. `.then_some()`, `.map()`, `.ok_or_else()`). +- Prefer iterator chains (`.filter_map()`, `.flat_map()`) over explicit loops for data transformation. ## Commit message style -- 2.51.2