# 読の樹 (yominoki)

A Japanese-learning app on atproto. Bluesky OAuth, JA posts off the Jetstream firehose, engagement (likes, follows, bookmarks) written back to the user's PDS as the signal. Most of the code is Japanese-NLP plumbing; the atproto layer is a self-contained, working reference for patterns still thin in public examples.

If you're building an atproto client, the auth layer is the part to read. If you're also learning Japanese, sign in and use it — PRs welcome.

## What's in here for atproto OSS devs

Everything below is in [`packages/atproto/`](./packages/atproto/src). ~530 LOC, no custom lexicons — all writes go to standard `app.bsky.*`.

| File | What you can lift |
| --- | --- |
| [`client.ts`](./packages/atproto/src/client.ts) | `NodeOAuthClient` factory. Dev uses a synthetic loopback `client_id` (`http://localhost?redirect_uri=…&scope=…`, no metadata fetch). Prod uses a confidential client with `private_key_jwt` + `jwks_uri` + DPoP. Single source for both. |
| [`oauth.ts`](./packages/atproto/src/oauth.ts) | `buildOAuthBundle({ baseUrl, jwksJson, ops })` — JWKS parsing, ES256 keyset import, loopback fallback. Works in Next.js (`process.env`) and the Cloudflare Worker (`Env` binding). |
| [`client-metadata.ts`](./packages/atproto/src/client-metadata.ts) | Production `client-metadata.json` shape served at `/client-metadata.json`. Pairs with a public `/jwks.json`. |
| [`stores.ts`](./packages/atproto/src/stores.ts) | `NodeSavedStateStore` + `NodeSavedSessionStore` over CockroachDB JSONB. Documents the warning you don't see elsewhere: `session.set` fires on every DPoP nonce rotation, and naive upserts under concurrent requests for the same DID will burn refresh tokens. Lock the row or accept the re-auth. |
| [`cookie.ts`](./packages/atproto/src/cookie.ts) | App session cookie. Carries only the DID as an A256GCM JWE (jose, `dir`). On each request: read DID → `oauthClient.restore(did)` → authenticated `Agent`. Refresh tokens never leave Postgres. |
| [`records.ts`](./packages/atproto/src/records.ts) | Write-throughs for `app.bsky.feed.like`, `app.bsky.graph.follow`, and bookmarks. Documents the gotcha that bookmarks are RPC-only on the AppView (`app.bsky.bookmark.createBookmark`) — they aren't repo records and need a different scope shape. |
| [`profile.ts`](./packages/atproto/src/profile.ts) | Unauthenticated `app.bsky.actor.getProfile` against `api.bsky.app` for sign-in backfill. |

The granular scope string in `client.ts` is a complete worked example of the [atproto permission spec](https://atproto.com/specs/permission):

```text
atproto
repo:app.bsky.feed.like?action=create&action=delete
repo:app.bsky.graph.follow?action=create&action=delete
rpc?lxm=app.bsky.bookmark.createBookmark&lxm=app.bsky.bookmark.deleteBookmark&lxm=app.bsky.bookmark.getBookmarks&aud=did:web:api.bsky.app#bsky_appview
```

### Bonus: a reusable Jetstream consumer

`apps/worker/` runs a Jetstream consumer in a Cloudflare Durable Object — a separate, reusable pattern. It connects to `wss://jetstream*.bsky.network/subscribe`, filters JA posts, splits sentences, and enqueues them for embedding. If you need a long-lived firehose tap on Workers without standing up your own infrastructure, it's a complete worked example.

## How it fits together

```mermaid
flowchart TB
    JS([Bluesky Jetstream])
    Cron([CF Cron])
    Firehose[apps/worker · Firehose DO]
    Jobs[apps/worker · Nightly jobs]
    D1[(D1<br/>tokenize_queue)]
    DB[(CockroachDB<br/>via Hyperdrive)]
    Runner[services/embed-runner<br/>Bun]
    Web[apps/web<br/>Next.js 16]
    Anth[Anthropic API<br/>Haiku grade / Sonnet retry]

    subgraph remote[Remote NLP — CF tunnel to colocated Mac]
        Kiri[kiri-yaiba<br/>tokenizer]
        Embed[embed-stack<br/>Qwen3-0.6B → 1024-dim]
    end

    subgraph atp[atproto network]
        PDS[(User PDS)]
        AV[Bluesky AppView<br/>api.bsky.app]
    end

    JS -->|JA posts| Firehose
    Firehose -->|enqueue URIs| D1
    Firehose -->|/ingest direct push| Runner

    Cron --> Jobs
    Jobs -->|GC, frequency, clusters| DB
    Jobs -->|follow graph| AV

    D1 -->|drain| Runner
    Runner --> Kiri
    Runner --> Embed
    Runner -->|tokens, embeddings| DB

    Web <-->|read corpus, fluency| DB
    Web -->|grade, etymology| Anth
    Web -->|likes, follows| PDS
    Web -->|bookmarks RPC, profile| AV
```

The Worker has two unrelated jobs in one deployable. The **firehose DO** filters JA posts and writes URIs to D1 `tokenize_queue` (or pushes directly to embed-runner for low latency) — its hot path never blocks on NLP. The **nightly cron** runs four idempotent maintenance jobs against Cockroach: GC of stale readings, token frequency rebuild, per-user topic clusters (k-means over vote-weighted embeddings), and a follow-graph snapshot per signed-in user from the AppView.

The embed-runner does the heavy NLP lifting in two phases — drain D1's `tokenize_queue` and call [kiri-yaiba](https://code.fugl.dev/kiri) for tokens, then drain Cockroach's `embedding_queue` and call [embed-stack](https://code.fugl.dev/embed-stack) for 1024-dim vectors. Both NLP services run on a colocated Mac behind a Cloudflare tunnel; neither the Worker nor the Next app loads a model in-process.

Engagement is a write-through: `apps/web` writes likes and follows to the user's PDS, bookmarks to the AppView (RPC), and mirrors them into Cockroach for fast reads.

## Repo

```
apps/
  web/             Next.js 16 (App Router, Turbopack) — surfaces + atproto OAuth routes
  worker/          Cloudflare Worker — Jetstream firehose DO + nightly cron jobs
packages/
  atproto/         OAuth client, JWKS bundle, Cockroach stores, JWE cookie, record writes
  ja-core/         Fluency state machine, time-decayed fluency, tokenizer client,
                   sentence splitter, morphology factoring, conjugation tables, embed client
  db/              Raw SQL migrations for CockroachDB + shared queries (no ORM)
  result/          Result<T, E> for boundary-level error handling
  ui/              shadcn/ui primitives
  eslint-config/, typescript-config/
services/
  embed-runner/    Bun process. Drains D1 tokenize_queue → kiri,
                   drains CRDB embedding_queue → embed-stack, writes both back to Cockroach.
```

## Conventions worth knowing

- **PDS is the source of truth for engagement.** Likes, follows, and bookmarks are written to the user's repo (or the AppView, for bookmarks). Cockroach mirrors them; it is not the system of record.
- **DID-only cookie.** The session cookie carries one claim. Tokens, DPoP keys, and refresh state live in `atproto_auth_session` and are loaded via `oauthClient.restore(did)` per request.
- **No ORM.** `packages/db` is hand-written SQL. Migrations are numbered files run in order.
- **`Result<T, E>` at boundaries.** Anything that crosses a process or network edge returns `Result`. Internal calls throw.
- **Remote NLP only.** If you're tempted to import a tokenizer into the Worker or web app — don't. Add an HTTP route to kiri-yaiba or embed-stack instead.
- **One `client_id` per environment.** Auth servers cache OAuth metadata aggressively. Dev (loopback) and prod (`https://host/client-metadata.json`) must not share.

## The app

Three surfaces, all backed by the same firehose-derived sentence corpus.

- **`/drill`** — Paired-direction sentence drill scoped to your interests and JLPT level (N5–N1). Voice input via Web Speech, register filter (any / 丁寧 / 普通), epsilon-greedy interest exploration. Claude Haiku grades each answer; flag the reference and Sonnet re-grades. Vote / like / bookmark / dismiss writes to your PDS.
- **`/n5`** — Daily treemap of N5 vocabulary. Tiles flip into a card with morphology factoring (stem/ending color-split), conjugation table, Wiktionary etymology accordion (Claude Haiku fallback when Wiktionary is empty), and Bluesky-sourced example sentences.
- **`/atlas/[level]`** — Full N1→N5 territory in three modes: saturation paint over time-decayed fluency, unknown-frontier repack, POS-grouped subdivision.

The opinionated bit: the unknown signal is per-token and time-decayed. Fluency isn't a JLPT level you pass; it's a state machine over each token, and the surfaces all read the same state.

## Contributing

Personal project, actively developed. PRs welcome from anyone along for the ride — particularly atproto devs who want to learn some Japanese in the process.

Useful first contributions:

- **atproto** — lexicon support for custom record types (not currently used, but the path is open)
- **ja-core** — more conjugation classes / edge cases in morphology factoring
- **ui** — better mobile layout for the `/atlas` views

Open an [issue](https://code.fugl.dev/yominoki/issues) first to align on direction.

## Run it locally

Bun 1.3+, Node 20+, a CockroachDB you can reach, an Anthropic API key, and reachable kiri-yaiba + embed-stack endpoints.

```bash
bun install
cp .env.example apps/web/.env             # fill in values
bun run --filter @yominoki/db migrate
bun run dev
```

Env access goes through `@/lib/env` in `apps/web` or a package-local `env.ts` — never `process.env` at a call site.

For local atproto OAuth: nothing extra. Dev uses a loopback `client_id` and skips JWKS — `bun run dev` and sign in.

For prod: a 32-byte `APP_COOKIE_KEY` (base64url) and an ES256 JWKS published at `https://<host>/jwks.json`. The matching private JWKS goes into `ATPROTO_PRIVATE_JWKS_PATH` (PEM bundle).

## License

MIT — see [LICENSE](./LICENSE).