Typeahead — repository instructions #

This root AGENTS.md is canonical. .claude/CLAUDE.md symlinks here so Claude and other agents use the same instructions. Paths are repository-relative.

documentation is part of the change #

• Before changing behavior, identify affected README sections, operational docs, public API docs, diagrams, and agent instructions. Update them in the same task as the implementation; do not wait for Nate to notice drift.
• Before calling a task complete, compare README.md against the final diff and verified deployment state. Update it, or explicitly state why no README change was needed. Architecture, runtime roles, data flow, schedules, deployment, reliability, and performance guarantees all trigger this review.
• Distinguish shipped behavior, local tested changes, experiments, and proposals. A successful local test does not make a feature live. Date operational status and link its evidence; do not leave historical estimates labeled current.
• Keep the overview in README.md, detailed mechanisms in docs/, and durable rules here. Link shared material rather than maintaining duplicate instructions.
• Preserve existing user edits. Documentation checks supplement engineering review; no automated check can prove a prose claim is true.

Community actor search for atproto. Discovery is the relay's firehose (com.atproto.sync.subscribeRepos), NOT jetstream — see docs/discovery-and-identity.md. Canonical endpoint: tech.waow.typeahead.searchActors (self-owned lexicon, lexicons/); app.bsky.actor.searchActorsTypeahead is served forever as the drop-in migration alias — the two must stay byte-identical (smoke enforces this). When endpoints change, update ALL public docs together: /docs, /llms.txt (src/pages/llms.ts), README.md, homepage API chip. Service identity: @typeahead.waow.tech (did:plc:y6kywgnb7xletjpa5hhxmd4k); lexicon publishing via scripts/publish-lexicon.py. README.md has the architecture; this file is the operational contract for assessing the system.

Before starting new work, read the git history — git log --oneline -30, then git log -p on anything that looks related. Commits are the record of what is in flight: they carry the date, the reasoning, and the code together, and they cannot drift from the tree the way a hand-maintained TODO list does. Use history together with the working diff and verified deployment state; committed, locally modified, and deployed code may differ. Resolve discrepancies and correct the docs rather than assuming one automatically describes production.

the serving invariant #

The target is predictable, bounded request work. The snapshot contains precomputed prefix lists and the merge caps hydration at 50 candidates before exact promotion. The deployed overlay lookup can still read every row for a matching prefix; do not claim constant or fully bounded work until measured and deployed. See the lookup evaluation for local experiments.

Avoid corpus-proportional request work and full-scan Worker cron selectors. Full index builds are deliberately corpus-scale offline jobs; this is not permission to move that work onto the request path.

performance & benchmarking #

Everything you need to assess performance is wired into the root justfile. Four layers, one command each; just perf runs them all in order. Results land in scripts/bench-results.json.

Defaults target prod (typeahead.waow.tech) and the search service (typeahead-search.fly.dev); smoke/bench-ranking/index-status take a positional url, bench/bench-search take --url X. Add --quick / --no-stress to bench/bench-search for a fast pass. bench-search skips stress by default so it doesn't hammer the single fly box.

The benchmark oracle is bluesky, not a fixed number. scripts/bench.py measures us side-by-side against public.api.bsky.app for the same queries every run, so the bar moves with the thing we're replacing rather than against a baseline that goes stale.

Acceptance criteria are two-tiered (acceptance_checks in scripts/bench.py):

• structural — must always pass. Wrong response, exact-handle mis-ranked (e.g. zzstoatzz.io must be rank 1 for q=zzstoatzz.io), display-name match missing, spam-dominated top 3.
• parity — bluesky's top result must appear in our top 8 for a named query set. These are expected to fail until authority-data coverage is real; that's deliberate, so PASS can't be bought by tuning lexical weights alone.

The canonical query set in bench.py is append-only by intent: once a query exposes a ranking bug, it guards against that bug forever (this is where the quality_score-clobber regression is pinned). Extend it, don't prune it.

when to run what #

• after any ranking / scoring / index change → local differential tests, then just bench-ranking + just bench-search against an explicit isolated target. Do not load-test production by default. For an optimization, require matching results against the old implementation on the same data, not just Bluesky parity.
• after a worker change → just smoke runs automatically as the post-deploy gate of just deploy-worker; run just bench if the change could touch latency or caching.
• after a snapshot rebuild / promote → just index-status to confirm the new build_id is serving, then just bench-ranking.
• periodic health → only when requested; do not start recurring monitors without authorization. just perf includes benchmarks, not just health probes.

deploys (smoke-gated) #

• just deploy-worker — wrangler deploy then the smoke suite. A deploy that breaks shape, ranking, or the admin API fails loudly here. deploy-worker-only is the escape hatch.
• just deploy-search / just deploy-ingest — the Zig fly roles (ingest + search).
• the snapshot builder (MODE=indexer) is NOT a Fly deploy. It runs on the home box (heavypad) as the typeahead-index Prefect deployment in the my-prefect-server repo, scheduled every 3 days (09:00 UTC): reads Turso → builds the prefix-index snapshot → uploads to R2 → App B promotes. To change it, edit flows/typeahead_index.py / prefect.yaml there and redeploy. prefect.yaml IS authoritative for schedules — each deploy replaces them. services/fly.indexer.toml is retained only as a manual rollback path. Host prereqs: deploy/home-indexer/install.sh. See docs/operations.md.
• typeahead-plc-identity — the other heavypad Prefect deployment, weekly (Mon 05:00 UTC). Bulk DID -> handle/pds from the PLC log, for what getProfiles cannot see. See docs/discovery-and-identity.md.
• Deploy is manual (not tangled CI — the secret path is unreliable; CI only builds + tests + wrangler deploy --dry-run). So the smoke gate lives in the deploy recipe, not in CI.

conventions #

• worker tests: bun test. zig tests: just test-services (or cd services && zig build test).
• never run work proportional to the corpus on the request/cron path (see the invariant above).
• the cron contract: every cron selector must be an index SEARCH (smoke's test_cron_selector_plans EXPLAIN-guards this live against Turso), every batch size must state what it honestly covers against measured arrival rates (~2,900 new actors/hour), and the cron NEVER claims to drain a corpus-scale backlog — backlogs belong to the bulk paths (the typeahead-enrich-backfill Prefect deployment for profiles, typeahead-identity-hourly for new actors' handles, typeahead-plc-identity weekly for what the appview cannot see — all on heavypad, defined in the my-prefect-server repo), run off-worker. Since 2026-09-03 the worker cron does NO enrichment at all: freshness, stats, and the moderation refresh only, inside a 12-minute budget (CRON_BUDGET_MS) so the 15-minute wall kill — which took 120 of 140 runs, silently — cannot recur. Every run logs cron_done with per-phase ms.
• every equality query on actors.handle needs COLLATE NOCASE, or it scans the whole table — idx_actors_handle is declared with that collation. Paginate on rowid.
• Turso is single-writer and the live ingester shares it. Bulk writes must batch (one statement per batch, not per row) and pace against observed latency.
• traffic stats: traffic_buckets counts search cache-misses by app; request_buckets counts EVERY answer the worker gives (app × endpoint × status, router wrapper in src/index.ts) and feeds the source · endpoint · status views on /stats. endpointLabel is a fixed label set — unknown paths must collapse to other, never mint a new endpoint. Check /stats?view=status first when something smells; a 16% 500-rate on /admin/ingest was invisible before this.
• ranking is by the stored actors.quality_score, not live followers; writes that touch aggregate counts must keep quality_score consistent with them (see the clobber regression guarded in bench.py + smoke.py).