From 1da42e986f3ec26ca5495020551316b5f148eb1d Mon Sep 17 00:00:00 2001 From: Jer Miller Date: Tue, 4 Aug 2026 16:30:36 -0600 Subject: [PATCH] docs(conversion): add the plate/strand/cable dictionary MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The conversion's boundary map now lives in this repo rather than outside it, because every implementation that has to build against these definitions can see the repo and nothing else. PORTING.md remains the porting doctrine — how to port; docs/conversion/ is the map — what the boundaries are, which end of each strand owns its contract, and what must be carried forward out of the Python. Three files plus a README carrying the vocabulary, the naming scheme, the one-contract-per-strand rule, contract tiers, write-new/read-old, the identity-beside-the-data rule with its three known instances, and the two reserved-word collisions that are live in shipped code. Deliberately definitional and present-tense. It carries no schedule, no status, no ownership of work and no decision history — those are tracked outside this repo, and anything of that kind appearing here is a defect. Within a boundary you own, correcting a fact or recording a carry-forward is expected. Adding, splitting or re-scoping a plate, changing a strand's definition or contract owner, or defining a cable are decided elsewhere. Co-Authored-By: Claude Opus 5 (1M context) --- docs/PORTING.md | 6 ++ docs/conversion/README.md | 95 +++++++++++++++++++ docs/conversion/cables.md | 33 +++++++ docs/conversion/plates.md | 168 +++++++++++++++++++++++++++++++++ docs/conversion/strands.md | 185 +++++++++++++++++++++++++++++++++++++ 5 files changed, 487 insertions(+) create mode 100644 docs/conversion/README.md create mode 100644 docs/conversion/cables.md create mode 100644 docs/conversion/plates.md create mode 100644 docs/conversion/strands.md diff --git a/docs/PORTING.md b/docs/PORTING.md index 20866abd4..edfd18d91 100644 --- a/docs/PORTING.md +++ b/docs/PORTING.md @@ -4,6 +4,12 @@ This document is for engineers and coding agents porting solstone behavior from Python into the Rust workspace under `core/`. It records the wave-0 rules before any behavior moves. +**The map is separate from the doctrine.** This document is *how* to port. The +boundaries themselves — every plate, every strand, which end owns each contract, +and what must be carried forward out of the Python — are defined in +[`conversion/`](conversion/README.md). Read that for what and where; read this +for how. + ## Workspace Scope The Rust workspace lives at `core/`. It contains a thin `solstone-core` bin, diff --git a/docs/conversion/README.md b/docs/conversion/README.md new file mode 100644 index 000000000..4e816d1b7 --- /dev/null +++ b/docs/conversion/README.md @@ -0,0 +1,95 @@ +# The conversion dictionary — plates, strands, cables + +**The canonical definitions of the journal's conversion boundaries.** If two pieces of work need to agree on what a boundary is called, what it holds, or which side owns its contract, it is defined here. + +This directory is **definitional and present-tense**. It says what the boundaries *are*. It deliberately carries no schedule, no status, no ownership of work, and no record of who decided what — those live outside this repo, and anything of that kind appearing here is a defect. + +**Companion:** [`../PORTING.md`](../PORTING.md) is the porting *doctrine* — how to port, layering, error mapping, data boundaries, JSON and hashing, version lockstep. This directory is the *map*. Read `PORTING.md` for how, read here for what and where. + +## The files + +| File | Holds | +|---|---| +| [`plates.md`](plates.md) | every plate, what it holds, its contract tier | +| [`strands.md`](strands.md) | every strand, its two plates, which end owns the contract, and what must be carried forward | +| [`cables.md`](cables.md) | whole use cases through several strands | + +## Vocabulary + +| Term | Means | +|---|---| +| **plate** | A boundary of the journal where strands connect from other plates. ⛔ A plate has no "sides" — strands can be bi-directional, and in/out gets confusing | +| **strand** | The minimum viable path connecting two plates, in Rust, robust. May be bi-directional. **The contract lives at one *end* of the strand, never both** | +| **cable** | A whole use case, from one major system to another, through several strands with plates between them | + +⛔ **"Surface" is not one of these.** It is reserved for **owner-visible** things. ⛔ Not "thread" either, so it never collides with the Rust threading model. + +## Naming + +| | | +|---|---| +| Plate | `P-name` — ⛔ never `P1`/`P2`; a number reads as a priority | +| Sub-plate | dashes add granularity within a plate: `P-journal-segment` | +| Strand | `S:plate:plate` — 🔴 **the second plate always owns the contract** | +| Cable | `C:plate:plate:plate…` — every plate in the use case, in order | + +Worked: `S:device-ingest:segment-media` — `segment-media` owns the contract. `C:device-ingest:segment-media:journal-segment`. + +## One contract per strand, at one end + +**The owner is the one-to-many end.** It must serve all comers and cannot negotiate per-peer. **Tiebreak when the strand is one-to-one: the durable end owns it.** By convention the owner is written second in the strand name. + +📌 *Why this is a rule and not a preference:* nearly every contract drift in this tree is two places owning one thing with nothing binding them — a relay string matched verbatim on both sides, a nonce alphabet hand-copied between languages, a prompt preamble held as a constant in one place and as a sha256 of itself in another, a frontend switching on phases its backend enum does not have. One owner per contract makes that class **unrepresentable** rather than merely detectable. + +## Contract tiers — by what kind of boundary it is + +| Tier | Is | Example | +|---|---|---| +| **types** | types in a binary | — | +| **schema** | an API or interface format | the OpenAPI contract, the callosum envelope | +| **fixture** | a durable storage format | the `x-journal-contract` files, the path-resolution vectors, the identity vectors | + +⛔ Not graded by enforcement strength — graded by the **kind** of boundary. A vector is a type of fixture. + +## Writing and reading + +🔴 **Write new, read old.** A new implementation does not have to write what Python wrote, as long as there is a contract for what it *does* write. When it **reads**, it stays compatible with the older format. Bit-identical writes are **not** required. ⛔ The one thing that is never acceptable is leaving older journal data unseen. + +🔴 **Identity is written beside the data, never re-derived from a name or a position.** Where a derived value doubles as a *lookup key*, writing it differently forks the namespace instead of just changing a format. **The answer is not to teach every reader two spellings — it is to stop deriving:** persist the identity next to the data, and let the derived string be a **label** that nothing resolves by. + +`device.json` in a segment directory is the worked example: the stream name became human-friendly precisely *because* it stopped being load-bearing. + +⚠ **Read-compat still applies to old data** — a reader handles records written before the identity was persisted. But **a writer never re-derives.** + +Known instances and what is left of each: + +| Instance | State | Residue | +|---|---|---| +| `entity_slug()` | `id` is written into `entity.json` on create (`think/entities/journal.py:59`, `:201`); `loading.py:86` reads `data.get("id") **or** entity_slug(name)` — the derivation is a **fallback** | the fallback branch, and the `entities/{entity_id}/` directory name | +| `ambiguity_id` | required on the row and hard-rejected when absent (`ambiguities.py:157-159` raises `missing ambiguity_id`) | essentially none | +| `sentence_id` | 🔴 **the real instance** — a 1-based ordinal recomputed at read time and stored nowhere | all of it | + +**The work: make the written identity mandatory, delete the derivation fallback, persist `sentence_id`.** + +## 🔴 Reserved words — two live collisions + +Two words mean an internal thing and an owner-facing thing at once, and **both collisions are already in shipped code.** + +| Word | ✅ Means ONLY | ⛔ Never means | Use instead | +|---|---|---|---| +| **health** | the **journal system's** health — is it running, is everything processed | the owner's physical health | **body** | +| **activities** | the **internal** facet activity model (`facets/{facet}/activities/{day}.jsonl`) | the owner's physical movement | **body motion · fitness · kinetics** | + +**All owner physiological data is `body`** — *body data*, *body records*, *body sources*. + +⚠ `apps/body/routes.py:1985` defines `_activity_analysis` over *wearable* activity while `think/activities.py` owns the internal model — one word, two meanings, live today. + +⛔ **Consequence for plate names:** `P-journal-health` is **system** health written durably. Owner body records are **`P-journal-body`**, a different plate. Never fold them. + +## Changing these files + +**The plate set and the strand definitions are fixed.** Within a boundary you own, correct a fact, refine a contract note, or record what must be carried forward — that is expected and welcome. + +⛔ **Do not, on your own initiative:** add, split, rename or re-scope a **plate**; change a **strand's** definition, tier, or which end owns its contract; define or re-scope a **cable**. Those are decided outside this repo. If the code contradicts a definition here, that is worth raising — say so plainly rather than quietly widening the definition to fit. + +⚠ **Every count and inventory here is a floor, not a census.** Two string-keyed dispatch registries resolve modules by name at call time with **zero static import edges** — `FORMATTERS` (`think/formatters.py:139-265`, reaches 12 modules) and `EDGE_SOURCES` (`think/edge_sources.py:39-73`, reaches 6), both via `import_module` + `getattr`. No import graph over this tree is complete. **A boundary is not clean until you count its callers.** diff --git a/docs/conversion/cables.md b/docs/conversion/cables.md new file mode 100644 index 000000000..ac0d156ff --- /dev/null +++ b/docs/conversion/cables.md @@ -0,0 +1,33 @@ +# cables — whole use cases, end to end + +**A cable connects a whole use case from one major system to another**, through several strands with plates between them. Named `C:plate:plate:plate…` — every plate in the path, in order. + +Definitions and vocabulary: [`README.md`](README.md). Siblings: [`plates.md`](plates.md) · [`strands.md`](strands.md). + +⚠ **A cable is a destination, not a work unit.** Its strands are. This file exists so the destinations stay visible while strands are built. + +--- + +## `C:device-ingest:segment-media:segment-sense:thinking:journal-segment` + +**A device's capture becoming a fully processed, thought-out result in the journal** — from device ingest, through sense media processing, through thinking, down to journal storage. The spine of the product. + +Strands: `S:device-ingest:segment-media` → `S:segment-media:journal-segment` → `S:segment-sense:journal-segment` → `S:segment-sense:thinking` → `S:thinking:journal-thinking` + +⚠ Every strand in this cable is Tier 1, so it completes when the core does. + +## `C:journal:index:web:cli-sol` + +**Journal content becoming answerable to an owner or a talent** — from journal storage, through the formatting layer, to the indexer, to the convey API, to the `sol` CLI. + +Strands: `S:index:format` → `S:journal:index` → `S:web:journal` → `S:cli-sol:web` + +--- + +## Candidate cables + +Recorded so they are not silently dropped. ⛔ None is authorized as a destination yet. + +- **link-to-trust** — `C:device-link:journal:web` — a device pairing and becoming visible to the owner as a trusted device. +- **body ingest** — `C:body-source:journal-body:web` — owner body data arriving and becoming readable. +- **honest state** — `C:system:system-health:journal-health:web` — the product reporting truthfully on itself, end to end. diff --git a/docs/conversion/plates.md b/docs/conversion/plates.md new file mode 100644 index 000000000..edf554aeb --- /dev/null +++ b/docs/conversion/plates.md @@ -0,0 +1,168 @@ +# plates — the boundaries strands connect to + +**A plate is a boundary of the journal where strands connect from other plates.** ⛔ It has no "sides" — the contract lives at one *end of a strand*, and by convention that is the second plate named in `S:a:b`. + +Definitions and vocabulary: [`README.md`](README.md). Siblings: [`strands.md`](strands.md) · [`cables.md`](cables.md). + +⛔ **Reserved words:** `health` = journal system health **only**; owner physiological data is **`body`**. `activities` = the internal facet model **only**; owner movement is **body motion / fitness / kinetics**. + +--- + +## `P-journal` + +The on-disk journal. 🔴 **Deliberately coarse.** Talk about `P-journal` at a high level; when doing work, talk about the sub-plate. + +| Sub-plate | Holds | +|---|---| +| `P-journal-segment` | the segment directory — chronicle day/stream/segment | +| `P-journal-stream` | stream declarations and state | +| `P-journal-thinking` | talent outputs and run logs | +| `P-journal-health` | durable **system**-health records and logs | +| `P-journal-body` | owner **body** records — never "health" | + +⚠ **6 formats carry a real `x-journal-contract`; roughly 20 more on-disk shapes are defined only by a Python writer**, 8 of them undocumented anywhere. The path spine itself — including the default-stream transposition that `layout.json` never states — is Python-only. + +## `P-device-link` + +Link identity, security, mTLS. + +⚠ **The machine-checked part of this boundary is the QR bootstrap only.** The `spl` conformance corpus covers pair-link forms `0x04`/`0x05`/`0x06`, address admission and ca_fp tags. The pairing ceremony, session lifecycle, framing and token JWTs are prose. + +🔴 **`convey/secure_listener/` is this plate's authorization boundary** — TLS termination, the in-handshake fingerprint check, admission, framing, mux, and inline WSGI dispatch. It is Python. + +**Identity:** the journal's own identity is a `jid` derived from its CA's P-256 SPKI. A **device's** identity is a **`did`** — the device certificate fingerprint, `sha256:` over **certificate DER** (`cert_fingerprint()`, `think/link/ca.py:269`), which is the value already stored per client. + +⛔ **Two kinds of fingerprint exist here on purpose and must never be crossed.** `cert_fingerprint()` hashes the **certificate DER** — that is the `did`. `spki_fingerprint_sha256()` (`ca.py:66`) hashes the **DER SPKI** and is the `0x06` pair-link CA pin; its own docstring says *"not cert DER."* Reversing them silently breaks off-LAN pairing. + +✅ The `did` is stable for the device's life — client certificates are 10-year and there is no renewal path in `think/link/`; revocation is via `authorized_clients.json`. ⚠ If a renewal path is ever added, a device's `did` changes and prior segments carry the old one. + +**Carry forward, non-negotiable:** the certificate fingerprint is the identity and the label is only a label · revocation is the ledger, not the certificate · **an unreadable `authorized_clients.json` authorizes nobody** · loading never rewrites the ledger. + +🔴 **The mark derivation is the most dangerous thing in this plate to reimplement.** `think/link/mark.py` calls itself the frozen journal identity contract. Argon2id v0x13, `t=3`, `m=65536`, `p=1`, `hash_len=32`, salt `solstone-journal-mark-v1`, secret = the jid's 16 raw bytes; seven big-endian u32 words drive `pick`/`pick_distinct` over 60 icons, 16 colours, 7,776 words, plus two rotation bits rendered `45 if bit else 0`. ⛔ **`_ICON_NAMES = tuple(_GLYPHS)` makes the icon index the JSON insertion order of `mark_assets/glyphs.json`.** That order is *currently identical to sorted order*, so an implementation that sorts passes every derivation vector and diverges the first time a glyph is appended out of alphabetical order. ⚠ The colour list is spectrum-ordered — that is the tell that file order is the rule. **The mark is what an owner looks at to confirm they are pairing with their own journal; a subtly-wrong reimplementation presents as evidence of compromise.** + +## `P-device-ingest` + +The ingest **API**. Deliberately separate from `P-segment-media`. ⚠ 9 published operations, 7 named client consumers. + +## `P-segment-media` + +An ingested, not-yet-processed segment. + +## `P-segment-sense` + +Media processing. ⚠ Emits **two** strands with different contracts — processed data back to storage, and callosum events out. + +## `P-index` + +The SQLite index. **Ephemeral by design and always rebuildable — that property is required, not incidental.** ⚠ The index schema needs architecture work. + +🔴 **Not already Rust.** `think/indexer/native.py:6-11` says only CLI command writes went native, that *"query and interactive search stay in Python,"* and names eight remaining in-process Python writers. `indexer/journal.py` is 1,693 live lines; `edges.py` is 1,263. + +## `P-format` + +Consistent formatting of **structured journal data** for its consumers — the indexer and the convey apps. + +🔴 **No import graph shows this plate's fan-out.** `FORMATTERS` (`think/formatters.py:139-265`) reaches 12 modules by **string key** via `import_module` + `getattr` (`:283-286`), with zero static import edges. It is the de facto read-side inventory of every on-disk shape, and it lives only in Python. + +## `P-thinking` + +🔴 **A grouping plate.** Holds **two contracts: `generate` and `cogitate`**. Everything connects to it. `P-local`, `P-BYO` and `P-SPP` sit behind it. + +⚠ The runtime preamble every talent is written against exists as a **sha256 only** in the cross-language fixture — drift is detectable, the text is not reproducible. + +## `P-local` + +Local model runtime, inside the security boundary. ~9k lines of live install/runtime machinery with zero dead modules. + +⚠ The boundary is loopback HTTP **plus a durable record** — `health/brain.json`, written and validated by `providers/brain_state.py` (2,692 lines, HMAC-fingerprinted, refresh lease, three lanes). Calling it a types boundary drops the durable half. + +⚠ **Do not carry** the in-loop USD ceiling — it fabricates a cloud price for local runs and force-stops them. Keep the **context**-fraction half of the same function. + +⚠ **Do not carry the loopback "guard" as written.** The `cmd` list is built with `"--host", "127.0.0.1"` hardcoded and the next statement is `if "0.0.0.0" in cmd: raise` — a membership test against a literal set three lines above. It cannot fire at runtime and would miss `--host=0.0.0.0`, `::`, or a hostname. ✅ **The honest statement of the same invariant is `services/spp_transport.py:190-197`** — it binds `("127.0.0.1", 0)` and writes down why same-UID reachability is acceptable and which constraints are actually load-bearing. Carry that one. + +## `P-BYO` + +An owner's own token to a known provider, or their own OpenAI-compatible URL and token. ⛔ **Egress.** Will split per provider. + +## `P-SPP` + +Confidential hosted processing. ⛔ **Egress.** Attested, non-retained. ⚠ **Fails closed by the absence of a fallback branch** — a refactor that tidies it can open a downgrade path with no test going red. Make "no downgrade path" an explicitly tested invariant. + +## `P-speaker-id` + +Per-statement embeddings → speaker fingerprints. ✅ A native kernel already sits behind an argv+stdio wire with an algorithm-identity handshake. + +⚠ `.npz` sidecar and `speaker_labels.json` are Python-only. ⚠ The `sentence_id` join key is recomputed at read time and stored nowhere. ⚠ The voiceprint corpus is real-person biometric data. 🔴 **Years of voiceprints must survive with no re-teach** — that is a shipped promise. + +## `P-entity` + +The entity store. ⚠ **47 distinct production modules** import `think.entities`, fanning in from at least four plates. + +🔴 **This store fails by BRICKING, not degrading** — the history tree and `ambiguities.jsonl` are self-validating, so a subtly-wrong implementation destroys the store rather than producing worse results. All three algorithm-identity hazards — `entity_slug()`, `casefold`, `rapidfuzz` at threshold 90 — live behind this boundary. + +## `P-facet` + +Facets and their per-facet contents, including facet-scoped entity and speaker material. ⛔ Distinct from `P-entity`: the entity store is the identity-bearing thing, the facet is the organizing structure over it. + +## `P-journal-config` + +`journal/config/journal.json`, read by **30 production modules**. Durable, `0o600`, mutated under `hold_lock` + `atomic_replace` with an explicit transaction type. + +✅ **Carry forward — this is the house style, not a local quirk.** `CorruptConfigError` (`think/utils.py:53-68`): a **missing** config returns deep-copied defaults; a config that **exists and will not parse raises**, in owner voice — *"I couldn't read your settings file… Your settings were NOT changed."* Two deliberately different postures on two failure modes, never silently substituting on the dangerous one. + +⚠ **The config file being the source of truth is an external commitment made in writing.** A contract-breaking pass here can violate it by accident. + +## `P-journal-retention` + +The logic that decides what raw media is retained, and what logs are retained for how long. `think/retention.py` (708 lines) **irreversibly deletes owner raw media**; `log_retention.py` (1,006) prunes logs. + +🔴 **Carry forward:** *read one extraction file strictly enough for irreversible deletion* (`retention.py:110`) — reads at most two lines and treats any `OSError` / `JSONDecodeError` / non-dict as **`"malformed"`, never as "empty, safe to purge"**, with an explicit guard at `:136-139` against a stray marker key making a header-only file look chunk-bearing. Plus `resolve_segment_gate`: `.npz` without `talents/speaker_labels.json` ⇒ incomplete. + +⚠ Retention imports `apps/backup/copy` and `think/offload.py` imports back out of retention, so deletion is coupled to backup. + +## `P-web` + +The journal web service — human interface, web apps, **and the API**. + +⛔ **Access model: 100% of `P-web` is either `localhost:5015` (human web, or a same-device CLI) or an authorized linked device. There is no third way in — the boundary *is* the authorization.** ⛔ A third access path is not a decision to make inside this repo. + +⚠ 77 of 135 routes uncontracted including 20 state-changing; `/api/shell` — the most load-bearing endpoint in the owner UI — is not in the contract. ⛔ Zero of 177 contracted operations declare authentication, so a port generated from the contract inherits an unauthenticated surface. + +## `P-CLI` + +Splits almost immediately. + +| | Reach | May do | +|---|---|---| +| `P-CLI-sol` | any device | **API only**, over a link | +| `P-CLI-journal` | the same journal device only | **modify the journal directly**, or the API over localhost | + +## `P-system` + +Operations for managing asynchronous activity — starting things, running things. + +**Carry forward:** the task-request refusal **classifies rather than guesses** — it distinguishes `"wedged"` (runtime > 2× cap) from `"still_running"` and emits a skip event with both refs and the reason. A refusal that says *which* refusal it is. + +## `P-system-health` + +Health of running things — current status, in-memory. ⛔ **System** health, never owner body data. + +🔴 The per-day health JSONL grammar is **entirely Python string literals**. The callosum envelope likewise — two constants and a docstring carrying the whole control plane. + +## `P-body-source` + +Owner **body** data arriving from outside — Oura, Apple Health. ⚠ **Ingress, not egress** — nothing of the owner's journal leaves. + +⚠ The normalized shard format is defined only by its **reader** (`apps/body/routes.py:311-334`), and `imports/health-dedupe.sqlite` uses raw `sqlite3` entirely outside `journal_io` discipline. 🔴 **Excluded from every backup with no rebuild path** — a restore silently empties the owner's body history. + +--- + +## ⛔ Egress — where the covenant applies + +Journal and devices are **one secure environment**. No per-plate privacy tracking inside it; transport security is already covered. + +**Actual egress — three:** `P-BYO` (the owner's own key, owner-directed) · `P-SPP` (attested, non-retained) · **support requests** — ⚠ the `_SECRET_*` redaction in `apps/support/diagnostics.py:29-50` is the **last thing** between a journal config and an external service. + +**Blind by construction, therefore not egress:** relay transit · push notifications · encrypted backups. + +🔴 **Push is only reimplemented in an end-to-end encrypted form** — the journal encrypting and the receiving device decrypting with the link cryptographic identities. ⛔ The current plaintext path, which carries journal-derived chat content to a push service and which unpairing does not revoke, does **not** come across. diff --git a/docs/conversion/strands.md b/docs/conversion/strands.md new file mode 100644 index 000000000..5742ef17c --- /dev/null +++ b/docs/conversion/strands.md @@ -0,0 +1,185 @@ +# strands — the work units + +**A strand is the minimum viable path connecting two plates, in Rust, robust.** It may be bi-directional; its contract lives at exactly **one end**, and by convention that end is written second in the name. + +Definitions and vocabulary: [`README.md`](README.md). Siblings: [`plates.md`](plates.md) · [`cables.md`](cables.md). + +**Read your strand's section, not the file.** Organized for surgical section edits. + +⚠ **Every count here is a floor, not a census** — see `README.md` on the two string-keyed dispatch registries that defeat counting. + +⛔ **Reserved words:** `health` = journal system health only; owner physiological data is `body`. `activities` = the internal facet model only. + +--- + +## Tier 1 — the core + +### `S:device-link:journal` +**Connects** `P-device-link` → `P-journal` · **Owner** `P-journal` · **Tier** fixture + schema + +Link identity. A device proves who it is; the journal decides whether it is authorized. + +⚠ **24 production (non-test) modules import `solstone.think.link`**, which makes it the least isolable thing in the tree — an argument for converting it early rather than deferring it. + +The identity derivation, the two fingerprint kinds, the `did`, and the mark's exact parameters are in [`plates.md`](plates.md) § `P-device-link`. ⛔ Do not restate them here. + +**Carry forward:** the certificate fingerprint is the identity, the label is only a label · revocation is the ledger, not the certificate · **an unreadable `authorized_clients.json` authorizes nobody** · loading never rewrites the ledger. + +### `S:device-ingest:segment-media` +**Connects** `P-device-ingest` → `P-segment-media` · **Owner** `P-device-ingest` · **Tier** schema + +Segments arriving from a device. This is the **ingest envelope** — a wire shape, not bytes on disk: `observe/protocol.schema.json` carries `file_kind: "ingest_envelope"` and `producer_write_paths` of the ingest endpoint. + +⛔ **The client no longer asserts its own identity.** The journal authenticates the certificate at the transport, derives the `did`, and records it. Nothing routes, authenticates, or names a stream from a client-supplied string. + +**Carry forward:** create-exclusive byte writes that never overwrite — on collision compare SHA-256 and record the already-held case · content identity **refuses rather than guesses** · the segment-key timezone semantics — `HHMMSS` is **device-local wall clock, not UTC**. + +⚠ The audited client bundle and the plate's declared operations are **different sets**. The bundle omits the owner's location-data delete route — the covenant-critical one — and adds three operations belonging to other plates, so the capture clients contractually depend on chat and callosum. + +### `S:segment-media:journal-segment` +**Connects** `P-segment-media` → `P-journal` · **Owner** `P-journal` · **Tier** fixture + +Raw media landing durably, and the **segment sidecars**. + +**`device.json`** — optional; when present it carries at minimum the **`did`**, plus optional ephemeral device metadata the device supplies (battery level, screen lock, posture) as **free-form fields opaque to the journal and validated only for the required `did`**. + +⛔ **`device.json` is journal-authored, never client-uploaded.** Reserved sidecar names are never written from client bytes and never appear in segment listings as held. The device supplies its fields as data over the API; the journal writes the file. + +🔒 **Nothing is ever looked up by stream name. Attribution always reads the segment.** The stream name is a human-friendly, unique, cross-platform-filesystem-safe label — a deterministic projection of a display name like `iPhone (2)` to `iPhone_2`. One device may own several streams (a watch app via a phone is a different stream). + +⚠ **The projection's uniqueness must be case-insensitive.** APFS and NTFS fold case by default; ext4 does not, so two names differing only in case collide on two of three platforms and not on the one most likely to be developing them. Also exclude `<>:"/\|?*`, trailing dots and spaces, and the Windows reserved device names. + +**Carry forward:** **unresolved error → hold raw**, via `exit 69` — the handler writes nothing, leaves the input, and the scanner records *neither* success nor failure. The deferral emitter deliberately swallows its own bus failure so a down bus cannot turn a deferral into data loss. + +### `S:segment-sense:journal-segment` +**Connects** `P-segment-sense` → `P-journal` · **Owner** `P-journal` · **Tier** fixture + +Processed sense output written back. + +🔴 `_solstone_processing` has a version string and **no schema**, and is absent from both sibling schemas that enumerate every *other* header key. ⚠ **Read by five planes** — the fifth is `think/retention.py:133`, the highest-stakes one, because it decides irreversible deletion. + +**Carry forward:** **terminal-empty written before the raw is unlinked** — die between them and the file survives with a terminal marker, never the reverse · atomic promote through a same-dir temp, header **last** · detections stored raw, filtered at read time. + +### `S:segment-sense:system` +**Connects** `P-segment-sense` → `P-system` · **Owner** `P-system` · **Tier** schema + +Callosum events emitted as processing happens. + +There **is** a published machine-readable registry — `CALLOSUM_REGISTRY` (`convey/contract/assemble.py:43-99`), emitted as `x-callosum-registry`, 11 tracts and ~60 events, plus two published SSE operations. ⚠ **It is already drifted three ways:** events produced but undeclared · events emitted and prose-documented but absent from the registry · events declared with no literal producer. ⚠ The drift count is a floor — tracts passed through variables are invisible to a literal grep. + +### `S:segment-sense:journal-segment-events` +**Connects** `P-segment-sense` → `P-journal` · **Owner** `P-journal` · **Tier** fixture + +The **durable** half of the callosum contract, split from the wire half above. The whole bus envelope is appended verbatim into `{day}/[{stream}/]{segment}/events.jsonl`, with readers in `think/segment.py` and `apps/observer/utils.py`. + +⚠ **That writer bypasses `journal_io`** — bare `open(…, "a")` — and swallows failures at debug level. ⛔ The segment-sidecar family wants one write discipline; do not let a new sidecar inherit this one. + +### `S:segment-sense:thinking` +**Connects** `P-segment-sense` → `P-thinking` · **Owner** `P-thinking` · **Tier** schema + +⚠ The talent event vocabulary is **NDJSON on stdout** — a real inter-process wire and the run-log format that gets persisted — defined as `total=False` TypedDicts with no schema, fixture, or validator. **The shape most likely to be lost silently.** + +### `S:thinking:journal-thinking` +**Connects** `P-thinking` → `P-journal` · **Owner** `P-journal` · **Tier** fixture + +Talent output landing durably. Carries the failure semantics: retries, back-offs, days being complete, segments being complete. + +**Carry forward:** the talent-use lifecycle where **the filename is the lock and the state** — exclusive `open(…, "x")` on `{use_id}_active.jsonl` is the claim, rename on completion, and on restart every leftover `_active` is terminalized with an error so an interrupted talent is never indeterminate. + +🔴 **`_active.jsonl` is not only a talent convention — it is a deletion gate in two other subsystems.** Seven production readers, including `think/log_retention.py:368` (skips pruning them) and **`think/retention.py:199` (treats presence as "segment incomplete, do not purge raw media")**. A cleaner claim filename is a format change on the writer side and a **silent capability loss on the deletion side.** + +### `S:thinking:local` +**Connects** `P-thinking` → `P-local` · **Owner** `P-thinking` · **Tier** schema + fixture + +The local model lane. ⚠ Not a types boundary — it is loopback HTTP **plus a durable record**. See [`plates.md`](plates.md) § `P-local` for the two things not to carry. + +### `S:journal:index` +**Connects** `P-journal` → `P-index` · **Owner** `P-index` · **Tier** fixture + +🔴 **Not already Rust** — see [`plates.md`](plates.md) § `P-index`. + +⚠ **Fan-in is nine writers across six plates**, not one: `think/backup/restore.py:31`, `convey/chat_stream.py:182`, `think/importers/cli.py:27`, `think/day_accumulator.py:12`, `apps/observer/prune.py:27`, `apps/observer/share_delete.py:19`, `think/entities/merge.py:2043`, `think/segment.py:19`. + +⚠ Plus an **invisible runtime dependency** on `apps/speakers/edges.py` and the entity store via the `EDGE_SOURCES` registry, so the index build runs `find_matching_entity` — the `rapidfuzz`-at-threshold-90 path. + +**Carry forward:** the index is **always rebuildable** — ephemeral by design, and an interrupted update never leaves a partial result. ⚠ That property is required, not incidental. + +### `S:index:format` · `S:web:format` +**Owner** `P-format` in both · **Tier** schema + +The indexer's and the convey apps' consumption of consistently formatted structured journal data. ⚠ `P-format` owns both because it is the one-to-many end — it serves all consumers and cannot negotiate per-consumer. ⛔ The name encodes contract ownership, not data flow. + +### `S:index:*` — the read / query path +**Owner** ⚠ unassigned · **Tier** schema + +Nine production readers across search, tools, voice, talents and connections. The `search_journal` / `search_counts` / `known_agents` interface — which `P-thinking` calls **directly, in-process, on every talent that searches** — belongs to no strand yet. + +### `S:*:P-entity` · `S:*:P-facet` +**Owner** `P-entity` · `P-facet` · **Tier** fixture + +See [`plates.md`](plates.md) — this store fails by **bricking**, not degrading. + +### `S:*:journal-config` +**Owner** `P-journal-config` · **Tier** fixture + +See [`plates.md`](plates.md) § `P-journal-config` for the fail-closed posture that is the house style. + +### `S:web:thinking` — chat +**Owner** `P-thinking` · **Tier** schema + +The primary owner-facing use of the model. `convey/chat.py` (2,532 lines) + `chat_stream.py` (512) + `convey/sol_initiated/` (1,034). It **spawns talents**, so it is a producer into `P-thinking`, and it is in the audited native-client bundle — the capture clients depend on it. + +⚠ **Push cannot be scoped until this exists** — push's trigger is the **chat tract** (`push/triggers.py:63-64`), so push is a callosum consumer downstream of the chat orchestrator, not a standalone journal→device path. + +### `S:*:system` — the command channel +**Owner** `P-system` · **Tier** schema + +`_handle_task_request` takes `message["cmd"]` off the unix socket and hands the argv to the task queue. **This is how the scheduler, importers, backup, and the sense/think pipeline all cause work to run** — six production producers. ⚠ Distinct from liveness/status. + +### `S:journal:establish` +**Owner** ⚠ needs assigning · **Tier** fixture + +First-run journal establishment. **Creates the identity root** that `S:device-link:journal` depends on: the mark-lock route promotes the staged CA and persists the instance identity. + +⚠ Eight `/init/*` routes are session-gate-exempt and admitted **before** the journal-is-active check. That is the same `localhost:5015` human-entry basis as everything else on `P-web`, before a session exists to gate — **not** a third access path. + +--- + +## Tier 1 — retention + +`P-journal-retention` connects through three strands, each a different contract: + +| Strand | For | Owner | Tier | +|---|---|---|---| +| `S:journal-retention:journal-config` | the **posture / settings** it reads | `P-journal-config` | fixture | +| `S:journal-retention:system` | **when it runs** | `P-system` | schema | +| `S:journal-retention:journal` | **tending the files** — changes, and recording status | `P-journal` | fixture | + +Where retention is the provider it owns the contract — it is the one-to-many end with ten production call sites. See [`plates.md`](plates.md) for the irreversible-deletion carry-forward. + +--- + +## Tier 2 + +| Strand | Owner | Note | +|---|---|---| +| `S:system:system-health` | `P-system-health` | Liveness and current status of running things | +| `S:system-health:journal-health` | `P-journal` | 🔴 Per-day health JSONL grammar is entirely Python string literals | +| `S:web:journal` | `P-journal` | ⛔ 100% of `P-web` is `localhost:5015` or an authorized linked device | +| `S:cli-journal:journal` | `P-journal` | Same-device only; may modify the journal directly | +| `S:cli-sol:web` | `P-web` | Any device, API only, over a link. ⚠ May share `S:web:journal`'s contract | +| `S:segment-sense:speaker-id` | `P-speaker-id` | A refinement of sense | +| `S:speaker-id:journal-facet` | `P-journal` | ⚠ **Years of voiceprints must survive with no re-teach** — a shipped promise. **Carry forward:** the label-source resolver returns *ambiguous* and warns the owner rather than picking a source | +| `S:body-source:journal-body` | `P-journal` | Owner **body** data — ingress, not egress. ⚠ The shard format is defined only by its reader; 🔴 excluded from every backup with no rebuild path | +| `S:file-import:journal-segment` | `P-journal` | Generic import. ⚠ **A second write door into `chronicle/`** that consults no contract and writes with bare `write_bytes` — the create-exclusive guarantee is a property of **one** of the two doors | + +## Tier 3 — additive, absent until approved in + +`S:thinking:byo` — ⛔ egress, the owner's own key · `S:thinking:spp` — ⛔ egress, attested. ⚠ Fails closed by the *absence* of a fallback branch; make "no downgrade path" an explicitly tested invariant. + +## Not yet placed + +- **push** — ⚠ cannot be scoped until `S:web:thinking` exists; its trigger is the chat tract. Only reimplemented end-to-end encrypted. +- **encrypted backup** — blind by construction, but ⚠ **retention imports it**, so a core deletion path depends on it. 🔴 The 64-char recovery key **is** the repository password. **Carry forward:** the exclusion-list comment records a paid-for lesson about basename-at-any-depth matching that a rebuild would otherwise re-learn; and `brain.json` / `scheduler.json` / the supervisor-ready marker are excluded deliberately, so post-restore provider and scheduler state is an intentional blank. +- **support request** — ⛔ real egress; the `_SECRET_*` redaction is the last thing before an external service. +- **merge / transfer** — deferred. ⚠ `think/merge.py:30-33` imports three **private** functions out of `think/entities/merge.py`, so the frozen thing depends on the unstranded store. -- 2.51.2