typeahead #

Community actor search for atproto.

Search · Stats · API docs · Machine-readable docs

curl 'https://typeahead.waow.tech/xrpc/tech.waow.typeahead.searchActors?q=nate&limit=10'

Returns the profileViewBasic fields without viewer state: did, handle, displayName, avatar, associated, labels, and createdAt. The self-owned lexicon defines the canonical endpoint. /xrpc/app.bsky.actor.searchActorsTypeahead remains a compatible alias with identical parameters and response.

Profile cards use /xrpc/app.bsky.actor.getProfiles?actors=… (up to 25 actors) or /xrpc/app.bsky.actor.getProfile?actor=…. They include available profile and enrichment fields; counts are omitted until enriched. No viewer state.

How it works #

flowchart LR
  Clients -->|search| Worker[Cloudflare Worker]
  Relay[Bluesky relay] -->|subscribeRepos| Ingester[Fly ingester]
  Ingester -->|batches and cursor| Worker
  Worker -->|actor writes and accounting| Turso
  Worker -->|cache misses| Search[Fly search]
  Turso -->|changed rows and tombstones| Search
  Turso -->|database export| Builder[Heavypad index builder]
  Builder -->|index.db and manifest| R2
  R2 -->|verified snapshot| Search

• Worker: public endpoints, a 60-second search cache, admin ingestion, request accounting, and profile APIs. Turso search fallbacks are opt-in. Bluesky is an explicit rollback option, not the normal search backend.
• Ingester: Zig on Fly, consuming the relay's subscribeRepos firehose (not Jetstream). Profile and activity events become batched actor updates. Failed writes retain their queues and apply backpressure; startup resumes from the persisted relay cursor. Identity events enter the protected discovery queue, and each one is re-resolved from its DID authority with the handle verified both ways, so a handle change lands within seconds. Watchdog exits and panics terminate on a deadman that does not depend on telemetry flushing.
• Turso: durable actor records, moderation overrides, and tombstones. Ingestion, enrichment, and request accounting share its writer.
• Index builder: the same Zig codebase in MODE=indexer, orchestrated by Prefect on heavypad every three days. It exports Turso, builds prefix-to-actor lists and compact profiles, and publishes a SQLite snapshot to R2. It is an offline batch job; an unsuccessful build leaves the previous snapshot serving.
• Search: Zig on Fly. It verifies and attaches the R2 snapshot read-only, then pulls newer Turso rows into a mutable overlay approximately every five minutes. Queries merge snapshot and overlay candidates, promote exact handles, hydrate profiles, and apply moderation. There is no separate full actor-table replica on the serving machine; the snapshot itself contains compact profiles.
• Maintenance: Prefect jobs on heavypad handle identity and profile enrichment, including weekly PLC processing for rows with no handle yet (they do not correct a stale handle; the ingester does). The hourly Worker cron handles freshness, statistics, and moderation refresh—not bulk enrichment.

The snapshot precomputes search candidates, but the deployed overlay lookup can still scan a large matching prefix. Candidate hydration is capped at 50 before exact-handle reinjection. Prefix keys are stored up to 16 bytes; a longer query looks up its 16-byte cut and the hydrated rows are then filtered against the whole query, so a long query costs what its cut costs plus one pass over at most 50 rows. A capped result list does not imply bounded SQLite work. See architecture and discovery and identity.

Public pages and metadata #

The Worker serves the homepage (/), /stats, /docs, /reputation, and /llms.txt, all from src/pages/ on one shared theme. Dated state:

• Homepage navigation (September 6, 2026): [src] and [llms.txt] share the project-resource row; the theme control sits at the right; attribution, stats, and docs are in the footer.
• Link previews (September 20, 2026): every page's description leads with what typeahead does and where it looks, and does not define the service by what it replaces. The card at /og-image (src/handlers/og.ts) reads the hourly stats KV blob, never Turso, and shows four cumulative figures: actors indexed with handle coverage, avatar coverage, apps that have ever sent a search, and searches answered all time. The app count is a filtered traffic_sources count the cron materializes as traffic_sources_total. Embedders cache cards by URL, so CARD_VERSION in src/pages/theme.ts must be bumped whenever the card changes.
• App metadata (September 20, 2026): the mark is served at /favicon.svg, rasterized at /apple-touch-icon.png, and declared in /site.webmanifest. A data: favicon preceded this; crawlers and home-screen installs could not read it.

"Hidden by moderation" left the card on September 20, 2026. That count is label observations from the refresh cron, not enforcement: PDS-level suspensions arrive as #account events and are deleted, and appview takedowns are re-enriched visible by policy. See operations.

Operational status #

As of September 6, 2026:

• Search runs on one Fly machine: 4 shared CPUs, 2 GiB RAM, and a 60 GB volume. Ingester runs separately on 1 shared CPU and 256 MiB RAM.
• The ingest retry/cursor fixes are deployed. Historical recovery restored and search-verified 377 accounts; remaining historical uncertainty is documented in the recovery closeout.
• Bulk profile enrichment is paused following writer contention. A guarded workflow is prepared locally in my-prefect-server; it is not deployed.
• The bounded overlay lookup went live on the serving machine on 2026-09-09 05:34 UTC: it was on main and shipped with the deadman exit-path deploy, ahead of the staged rollout the canary was preparing. /debug/search reports overlay_bounded: true on union keys. Before that it had passed the shadow canary's 18/18 live-query comparisons, all 36 full-overlay query cases (11.16M prefix rows) and 108 concurrent-update comparisons. The ranking index is NOT installed in production: it adds 543 MiB and increases projection write work, and without it SQLite still visits and sorts every matching prefix row before the bound applies. The stopped canary machine and its temporary 60 GB volume were deleted on 2026-09-28; evaluation evidence is preserved and the ranking-index experiment remains deferred. See the evaluation and canary limits and rollback.

These are dated observations, not configuration sources. Fly role configs live in services/; Prefect schedules and deployment definitions live in my-prefect-server.

Development #

bun install
just check             # Worker types, lint, tests, deployment dry-run
just test-services     # Zig unit tests
bunx wrangler dev      # local Worker

The Zig binary requires MODE=ingester, MODE=search, or MODE=indexer. There is no default role. Use the role's configuration and credentials; a bare zig build run is not a configured local service.

• src/ — Worker handlers, public pages, and shared page theme.
• services/ — Zig ingester, search, and index builder.
• deploy/home-indexer/ — heavypad prerequisites.
• scripts/ — smoke checks, benchmarks, recovery tools, and offline experiments.
• ops/ — health exporter and dashboard tooling.
• docs/ — architecture, operations, evaluations, and dated retrospectives.

Repository instructions live in AGENTS.md. .claude/CLAUDE.md is a symlink to that same file.

Deployment and validation #

just deploy-worker     # Cloudflare deploy, then smoke checks
just deploy-ingest     # Fly ingester
just deploy-search     # Fly search

These commands affect production. Search currently has a single serving machine; use an isolated target for performance experiments and account for availability before replacing it. Benchmarks and smoke scripts may default to production; check their target and whether they perform writes before running them.

The home builder is deployed through Prefect. services/fly.indexer.toml remains an optional manual Fly rollback path, not the scheduled builder. See operations.

Costs #

The existing hub cost snapshot reports project-attributed line items and their estimation basis. Turso and Cloudflare subscriptions are shared with other projects; their whole account invoices are not Typeahead's cost. COSTS.md retains dated historical research; do not use its older resource sizes or totals as current estimates.