find-bufo #

hybrid semantic + keyword search for the bufo zone

live at: find-bufo.com

overview #

a one-page application for searching through all the bufos from bufo.zone using hybrid search that combines:

• semantic search via multimodal embeddings (understands meaning and visual content)
• keyword search via BM25 full-text search (finds exact filename matches)

architecture #

• backend: zig (server/, std.http.Server) — fly app find-bufo-zig, serving find-bufo.com since 2026-08-28. the rust backend in src/ is retired (its fly app is gone) and kept only until the rust code is removed
• consumers: see docs/consumers.md — who calls what, and what they read back
• observability: see docs/observability.md — what the backend emits to logfire and the queries for traffic questions
• frontend: vanilla html/css/js
• embeddings: voyage ai voyage-multimodal-3
• vector store: turbopuffer
• deployment: fly.io

setup #

1. install dependencies:

   • zig 0.16
   • python 3.11+ with uv (ingestion only)
   • just, fly (deploys)

2. copy environment variables:

   cp .env.example .env
   

3. set your api keys in .env:

   • VOYAGE_API_TOKEN - for generating embeddings
   • TURBOPUFFER_API_KEY - for vector storage

ingestion #

to populate the vector store with bufos:

just re-index

this will:

1. scrape all bufos from bufo.zone
2. download them to data/bufos/
3. generate embeddings for each image with input_type="document"
4. upload to turbopuffer

development #

run the server locally (reads .env, serves ../static):

just run          # http://localhost:8799
just test         # server/ unit tests
just deploy       # cross-compile + fly deploy find-bufo-zig

the backend code is in server/ (see server/justfile); the bot in bot/.

deployment #

deploy to fly.io:

fly launch  # first time
fly secrets set VOYAGE_API_TOKEN=your_token
fly secrets set TURBOPUFFER_API_KEY=your_key
just deploy

for a frontend-only change, ship static/ without rebuilding the backend:

just stamp-static           # refresh ?v= hashes in index.html, then commit
just server::deploy-static  # layers static/ over the image prod already runs

usage #

1. open the app
2. enter a description, or upload, paste, drop, or take a photo
3. see the top matching bufos with hybrid similarity scores
4. click any bufo to open it in a new tab

api parameters #

the search API supports these parameters:

• query: search text (required)
• top_k: number of results (default: 10)
• alpha: semantic weight in rank fusion (default: 0.5)
  • 1.0 = pure semantic (best for conceptual queries like "happy", "apocalyptic")
  • 0.5 = default hybrid — a keyword rank-1 hit ties semantic rank-1, so exact names always surface
  • 0.0 = pure keyword (best for exact filename searches)
• formats: comma-separated file extensions to allow (default: all)
  • e.g. formats=png,jpg excludes gifs — useful for clients that can't render animations
• exclude / include: comma-separated regex patterns on the name; include overrides exclude
  • the 4x4 bigbufo_* tile set is excluded by default; include=bigbufo opts back in

example: /api/search?query=jumping&top_k=5&alpha=0.5

image similarity #

POST /api/similar?top_k=20 accepts raw PNG, JPEG, GIF, or WebP bytes. images may be up to 10 MB, are used only to produce the query embedding, and are not stored. upload responses use Cache-Control: no-store.

curl --data-binary @frog.jpg \
  'https://find-bufo.com/api/similar?top_k=5'

GET /api/similar?url=<image-url>&top_k=20 finds bufos visually similar to an existing first-party result image and gives that search a shareable URL. the source image is omitted from the response. only HTTPS images hosted by all-the.bufo.zone or find-bufo itself are accepted; top_k is capped at 100.

example:

/api/similar?url=https%3A%2F%2Fall-the.bufo.zone%2Fbufo-dolly.png&top_k=5

identifying your app #

set an X-Client header (ideally your domain, e.g. my-app.example.com) so your app shows up by name in the traffic breakdown on the bot stats page. without it, requests are attributed by Origin, then Referer, then counted as unknown — so browser apps are usually attributed anyway, while server-side, CLI, and native callers are not. this is the same convention as typeahead.waow.tech.

fetch('https://find-bufo.com/api/search?query=lgtm', { headers: { 'X-Client': 'my-app.example.com' } })

how it works #

ingestion #

all bufo images are processed through early fusion multimodal embeddings:

1. claude haiku sees up to 5 sampled frames and writes a one-sentence description plus concept tags (cached in scripts/captions.json)
2. filename text (e.g., "bufo-jumping-on-bed" → "bufo jumping on bed") + caption + one representative frame (alpha-composited on white, capped at 256px) go into a single voyage-multimodal-3 request — exactly one image per document, because extra frames or giant images inflate similarity to every text query
3. uploaded to turbopuffer with BM25-enabled name and caption fields for keyword search

every document also records the pinned caption model, caption prompt version, embedding model, and embedding-pipeline version. scripts/captions.meta.json binds the caption cache to its model and prompt, so changing either fails closed instead of silently mixing generations.

search #

1. semantic branch: query embedded using voyage-multimodal-3 with input_type="query"
2. keyword branch: BM25 over names and captions, each doc keeping its best score
3. fusion: weighted reciprocal rank fusion (k = 60)
   • score = α / (k + rank_semantic) + (1-α) / (k + rank_keyword)
   • ranks, not raw scores, so BM25 and cosine scales never mix; ties go to the keyword leg
4. ranking: results sorted by fused score (top result normalized to 1.0), top_k returned

why hybrid? #

• semantic alone: misses exact filename matches (e.g., "happy" might not find "bufo-is-happy")
• keyword alone: no semantic understanding (e.g., "happy" won't find "excited" or "smiling")
• hybrid: gets the best of both worlds

bot processing health #

https://bufo-bot.fly.dev/health/freshness returns 200 after Jetstream event handling completes, and 503 before the first completion, after five minutes without one, or when the subscription exits. It uses process-local monotonic time, not persisted statistics. Every event counts, including posts deliberately ignored by the bot; a quiet match or reply stream is healthy. This detects a stalled consumer, not the correctness of every reply or downstream write. The endpoint is read-only and is not a Fly routing or restart probe.