This repository has no description
ideonomics.app docs 08-implementation-notes.md
7.1 kB
Markdown
at dev

08 · Implementation Notes #

Where the running code deviates from, simplifies, or defers the design in 01–07. The honesty conditions (01) apply to the implementation too: these are the places where our reading of the spec could contaminate the data, recorded so they can be upgraded deliberately rather than forgotten.

Running it #

cargo run

Boots on http://127.0.0.1:8080 with zero configuration: local mode (see below), heuristic glosser, hashing embedder, SQLite at ./ideonomics.db.

Environment:

Var Default Effect
IDEO_BIND 127.0.0.1:8080 listen address
IDEO_PUBLIC_URL http://127.0.0.1:8080 OAuth client_id base; a loopback URL selects the atproto localhost-client convention, anything else serves /client-metadata.json
IDEO_DB ideonomics.db SQLite path (AppView + auth store)
JETSTREAM_URL wss://jetstream2.us-east.bsky.network/subscribe firehose endpoint
APP_ATPROTO_HANDLE / APP_ATPROTO_PASSWORD unset app account (app password) that owns square + gloss records
ANTHROPIC_API_KEY unset enables the LLM gloss ensemble
GLOSS_MODELS claude-haiku-4-5-20251001,claude-sonnet-5 comma-separated ensemble; a single model becomes a temperature ensemble
EMBEDDINGS_URL / EMBEDDINGS_API_KEY / EMBEDDINGS_MODEL unset OpenAI-compatible embeddings for canonicalization
SLOW_LOOP_INTERVAL_SECS 86400 refit cadence (also runs once at startup)

User login is real atproto OAuth regardless of the modes below: completions, resonances, and snapshots are always written to the user's own PDS.

Local mode (no app account) #

Squares and glosses are app-authored records (03). Without APP_ATPROTO_HANDLE, they are instead indexed AppView-only under at://did:web:localhost/... URIs with content-addressed CIDs (CIDv1/raw over the record JSON — content-addressed, but not repo commits). StrongRefs from user completions still validate and the whole pipeline functions; the audit property of 03 (third parties can fetch the exact square asked) is the one thing lost, restored the moment an app account is configured.

Degraded interpreters #

  • No ANTHROPIC_API_KEY → the ensemble is heuristic-v0, a deterministic structural interpreter whose gloss of a : b :: c : d is essentially "a ↔ b". It exercises canonicalization, the graph, and the renderer, but the readings are not semantic; every gloss record carries the interpreter id, so this data can be re-glossed or excluded later.
  • No EMBEDDINGS_URL → canonicalization embeds gloss text with a deterministic character-trigram feature hash (256-dim). Clusters are coarser than with a real embedding model; the relation vocabulary still grows organically and the threshold (0.55 cosine) was tuned against it.

Statistical simplifications (docs/04, 05) #

  • Mixture model. The "DP-flavored" hierarchical mixture is a greedy threshold clusterer over user relational profiles (sparse term|relation vectors + response-mode propensities) with centroid refinement, nonparametric in K (cap 12). Posteriors are softmax over scaled cosine similarity rather than true responsibilities. Entropy of the weights is still the contested-quilting measure.
  • Master signifier. Flagged as the top eigenvector-centrality term. The Boutyline–Vaisey screening-off/model-comparison test (05, family 1) is not yet implemented; the ingredients (per-user graphs, held-out likelihood machinery) exist.
  • Verdicts. quilted / dimensional / deflationary / insufficient are decided by an explicit documented score over gini × largest-component fraction, empty-signifier correlation, chain entropy, cycle ratio, and the factor baseline's held-out advantage. The component-fraction term exists because centrality Gini alone is inflated by fragmented graphs — hubs are cheap (05). All inputs are surfaced in the sky's epistemic frame.
  • Evaluation harness. Three baselines predict held-out canonical relations: population frequency, the quilt mixture, and a rank-2 factor model (power iteration — the compass in linear-algebra clothing). Held-out relations outside the training vocabulary hit an explicit OOV bucket; when OOV dominates (eval.in_vocab_frac < 0.5) the factor advantage is not allowed to drive verdicts. The parallelogram null as a completion-token predictor needs a real embedding model and is not yet a baseline.
  • Cycle suppression. Cycle rank is measured against edge count, not a degree-preserving null (05 notes the stronger form). Frustration index is a greedy 2-coloring, not exact.
  • The retroactivity probe (05, family 2 — the only prediction separating quilting from hub-ness) is not implemented: it requires scheduled re-presentation of squares after chains cross the nodal point. The chain parentage needed to analyze it is recorded on every completion.

Instrument details #

  • Latency is currently measured server-side (form render → submit) and bucketed; docs/02 asks for client-measured. A JS enhancement can override the hidden field without changing the record shape.
  • Chaining follows construction step 2: the chained square folds the user's last completion into a bank template chosen at random (probability 0.5 per serve). Chain-aware selection (expected-information-gain over chained candidates) is the natural next step in infer::fast.
  • Resonance rivals are other users' completions of the same square; archetype-generated rivals (02) await the LLM bank generator.
  • Bank generation. The seed bank (bank/seed_squares.json, 94 items) is hand-curated to the 02 criterion (quilt-incompatible relations + profane terms). The generative pipeline (archetype role-play, divergence filtering) is not yet built; bank.generator records provenance for when it is.
  • Comparison consent is explicit pairing only (request + accept). "Mutuals" (06) would need the Bluesky social graph and is not consulted.
  • XRPC. The AppView exposes no XRPC endpoints yet (07 flags this as eventually necessary); the HTTP surface is axum + HTMX only.
  • Snapshots embed the layout JSON in the record (nodes with polar coordinates, edges, verdict, fit) — renderable deterministically by any client, no blob needed.

Operational notes #

  • One binary, three task groups (07): axum server; Jetstream consumer + gloss worker (durable queue in SQLite, Notify for wake); slow-loop scheduler.
  • The write path mirrors optimistically (read-after-write); Jetstream re-delivery is idempotent by AT-URI upsert. Deletes propagate: completions hard-delete, dependent glosses flagged orphaned and excluded from graphs but retained for aggregates (03).
  • Layout cache key = SHA-256 over (completion URIs, gloss count, latest refit touching the user, layout version) — refits invalidate skies.
  • jacquard is pinned to =0.12.1 (pre-1.0; see 07 caveats). The reqwest major must match jacquard's (0.12) or its HttpClient impl doesn't apply.