typeahead at the race
on September 4, bsky38.com opened voting for 38 influential Bluesky posters. its account picker uses typeahead, so a lot of people deciding who to vote for became a lot of people querying my little actor index.
i think we held up pretty well! search volume went from 163 to 5,423 requests per 15 minutes, while the engine's median stayed around 3–4 ms. here's the afternoon, including the less flattering bits. you can scrub through it or play it back.
An afternoon of searches
September 4, 2026 · two 15-minute windows
Searches in 15 minutes
About 33 times as many searches
Typical time inside the search engine
More searches, with similar time spent answering each one.
About these measurements
These count searches, not individual people. “Typical” is the median: half of searches finished faster, half slower. Engine time excludes the rest of the trip from your browser. This comparison is an observed afternoon, not a capacity test.
Source: completed Logfire search spans, before sampling was enabled. All times are UTC. The table includes every complete 15-minute window from 16:45 to 19:00.
| Window starts | Searches | Median (ms) |
|---|
these are engine measurements from Logfire. the request still has to get from your browser through Cloudflare to Fly and back. the quieter baseline is 16:45–17:00 UTC; the 33× comparison uses that window and 18:30–18:45. the chart has finer, five-minute buckets.
my previous post was about replacing request-time full-text search with a precomputed prefix index. it had about 5.9M actors then. the snapshot we're serving now has 11.63M. same four shared CPUs and 2GB of RAM on the search machine.
how it runs now
since May, the heavy index builds have moved home, and the search machine has shed its full actor replica. the replica had survived the first redesign, even though the new snapshot already contained what we needed to answer queries. maintaining both was expensive, and occasionally wrong.
the search box now holds the snapshot plus an overlay of recent changes. below, search follows recorded queries through the index. profile update shows how a rename reaches that overlay. snapshot build follows the batch job from Turso, through heavypad, into R2 and back onto the search machine.
When someone changes their name
A small example of the prepared index and its recent changes.
What a search finds
alexAlex
ariNo match
Alex is in the prepared index. Try changing the display name.
Illustrated account and simplified name matching. “Recent changes” is the overlay used by typeahead.
try germnetwork.com in the search view. the d: lookup finds subdomains like tessa.germnetwork.com, which was one of the reasons for building this index in the first place. the lists are prepared ahead of time; at request time we merge them and hydrate at most 50 candidates.
updates were where keeping the replica got awkward. say someone's display name changes from Alex to Ari. we need to remove the prefixes for the name currently in our search index. a newer row in another database can tell us “Ari” and lose the information needed to remove “Alex”. reading the old value from the overlay, then the snapshot, gives us the representation we're actually replacing.
Turso is still the durable store. the ingester consumes the relay's subscribeRepos firehose and sends batches through the Worker into Turso. search projects changes into the overlay every five minutes. request-indexing can also push a row directly into the overlay after writing it, so someone explicitly asking to be indexed doesn't have to wait for that pass.
heavypad runs the builds through Prefect. it exports Turso, builds the SQLite index with Zig, and uploads it to R2. the search machine checks the new artifact before attaching it. September 4's build took about 67 minutes; a late build leaves the previous one serving. identity and profile enrichment run on heavypad too.
the remaining coupling
i'd left a fallback in the Worker that tried Turso's full-text search when the engine found nothing. it was meant to cover replica lag. the replica was gone, but the fallback was still there.
under the race traffic, those queries saturated Turso and interfered with ingest writes. the backlog overflowed and events were dropped. turning the fallback off fixed that, until a search-machine restart triggered another fallback for an unreachable engine. that one is off now too. during a restart, uncached requests can get an empty answer for a few seconds.
request counters still write to Turso, so there is more to separate here. and the events we dropped need partial recovery through enrichment and PLC discovery. the incident report has the timeline, including a separate 22-hour ingest hang caused by an unbounded telemetry flush.
what i'm paying to keep
the served artifact is now 14GB. removing the replica let us replace the search machine's 100GB volume with a 60GB one. ingest runs on one shared CPU, 256MB, with no volume. batch compute happens at home.
my May estimate was too optimistic about the serving bill staying flat as the corpus grew. index size and the memory needed to keep frequently used pages cached still grow. but this is a much more manageable setup, and i'm happy to have real traffic to measure it against.
i also made the race to 38, a live tracker for the vote. its “watch your votes” picker uses the same endpoint. the original bsky38 voting site is what drove the burst above.