diff --git a/.coveragerc b/.coveragerc index 58342c6c0..d94c42409 100644 --- a/.coveragerc +++ b/.coveragerc @@ -1,8 +1,8 @@ [run] source = - observe - think - convey + solstone/observe + solstone/think + solstone/convey muse omit = .venv/* diff --git a/AGENTS.md b/AGENTS.md index 6cbec2e4d..a8bd58c95 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -4,8 +4,8 @@ This file is the **developer guide** for the solstone repository. Read it before Audience: -- **Coders** (cwd = repo root, editing `observe/`, `think/`, `convey/`, `apps/`, `talent/`, `tests/`) — you're in the right place. -- **Cogitate talents** (cwd = `journal/`, running inside the live system) — your entry is `talent/journal/SKILL.md`, installed into `journal/.claude/skills/journal/` and `journal/.agents/skills/journal/`. +- **Coders** (cwd = repo root, editing `solstone/observe/`, `solstone/think/`, `solstone/convey/`, `solstone/apps/`, `solstone/talent/`, `tests/`) — you're in the right place. +- **Cogitate talents** (cwd = `journal/`, running inside the live system) — your entry is `solstone/talent/journal/SKILL.md`, installed into `journal/.claude/skills/journal/` and `journal/.agents/skills/journal/`. - **Operators** debugging a running system — see `docs/DOCTOR.md`. For the journal-side runtime entry point, see `journal/AGENTS.md`. @@ -17,13 +17,13 @@ For the journal-side runtime entry point, see `journal/AGENTS.md`. Read, in order, when you enter the repo for a coding task: 1. **This file through §8** — the invariants must be in working memory before your first edit. -2. **`think/sol_cli.py`** — the CLI entry point. Skim the `COMMANDS`, `ALIASES`, and `GROUPS` dicts. ~340 lines, scannable in one pass. You now know the whole top-level command surface. -3. **`think/top.py` (first ~100 lines)** — the interactive TUI. Ties callosum + supervisor + service status together in one vantage point. Good "oh, this is how it connects" moment. +2. **`solstone/think/sol_cli.py`** — the CLI entry point. Skim the `COMMANDS`, `ALIASES`, and `GROUPS` dicts. ~340 lines, scannable in one pass. You now know the whole top-level command surface. +3. **`solstone/think/top.py` (first ~100 lines)** — the interactive TUI. Ties callosum + supervisor + service status together in one vantage point. Good "oh, this is how it connects" moment. 4. **The area you're about to touch:** - - User-visible feature or `sol call ` → `apps//call.py` + `apps//routes.py` + `apps//templates/`. - - Think pipeline → `think/.py` + its tests. - - AI talent prompt or behavior → `talent/.md` (+ optional `.py` post-hook). - - Capture / observe → `observe/.py`. + - User-visible feature or `sol call ` → `solstone/apps//call.py` + `solstone/apps//routes.py` + `solstone/apps//templates/`. + - Think pipeline → `solstone/think/.py` + its tests. + - AI talent prompt or behavior → `solstone/talent/.md` (+ optional `.py` post-hook). + - Capture / observe → `solstone/observe/.py`. 5. **Run `sol`** (no args) — prints current journal status + grouped command list. Orients you to live state. 6. **`make dev`** or **`make sandbox`** when you need a running stack to iterate against. @@ -33,16 +33,16 @@ Read, in order, when you enter the repo for a coding task: | Dir | Purpose | Go here when | Depth doc | |-----|---------|--------------|-----------| -| `think/sol_cli.py` | CLI entry point — `COMMANDS` / `ALIASES` / `GROUPS` dicts | adding a top-level `sol ` | `docs/SOLCLI.md` | -| `observe/` | Multimodal capture — screen, audio, transcribe, describe, sense, transfer | capture-side bugs, new input modalities | `docs/OBSERVE.md` | -| `think/` | Post-processing core — cortex, talent, callosum, indexer, entities, facets, activities, scheduler, heartbeat, supervisor | anything downstream of capture; most coder work lives here | `docs/THINK.md`, `docs/CORTEX.md`, `docs/CALLOSUM.md` | -| `convey/` | Web app framework — app discovery, routing, bridge | layout / framework-level UI changes | `docs/CONVEY.md` | -| `apps/` | Convey apps — each self-contained (`call.py` Typer sub-app + `routes.py` + `templates/`) | adding a user-facing feature, a `sol call ` verb, a UI surface | `docs/APPS.md` (required reading before modifying `apps/`) | -| `talent/` | AI talent configs (markdown prompts + optional `.py` post-hooks) + `SKILL.md`s (journal, coder, partner, …) | defining or tuning a talent; adding a journal-side skill | `talent/journal/SKILL.md`, `docs/PROMPT_TEMPLATES.md` | +| `solstone/think/sol_cli.py` | CLI entry point — `COMMANDS` / `ALIASES` / `GROUPS` dicts | adding a top-level `sol ` | `docs/SOLCLI.md` | +| `solstone/observe/` | Multimodal capture — screen, audio, transcribe, describe, sense, transfer | capture-side bugs, new input modalities | `docs/OBSERVE.md` | +| `solstone/think/` | Post-processing core — cortex, talent, callosum, indexer, entities, facets, activities, scheduler, heartbeat, supervisor | anything downstream of capture; most coder work lives here | `docs/THINK.md`, `docs/CORTEX.md`, `docs/CALLOSUM.md` | +| `solstone/convey/` | Web app framework — app discovery, routing, bridge | layout / framework-level UI changes | `docs/CONVEY.md` | +| `solstone/apps/` | Convey apps — each self-contained (`call.py` Typer sub-app + `routes.py` + `templates/`) | adding a user-facing feature, a `sol call ` verb, a UI surface | `docs/APPS.md` (required reading before modifying `solstone/apps/`) | +| `solstone/talent/` | AI talent configs (markdown prompts + optional `.py` post-hooks) + `SKILL.md`s (journal, coder, partner, …) | defining or tuning a talent; adding a journal-side skill | `solstone/talent/journal/SKILL.md`, `docs/PROMPT_TEMPLATES.md` | | `scripts/` | Repo maintenance scripts — `check_layer_hygiene.py` | tooling that guards the codebase; wired into `make ci` | (none) | | `tests/` | Pytest suites + `tests/fixtures/journal/` mock journal | writing tests; debugging flakiness; `make dev` / `make sandbox` use fixtures as the journal | `docs/testing.md` | | `docs/` | All longform documentation | reference lookups; never your first stop | §10 below | -| `journal/` | The live journal (user data). Git-ignored content; checked-in template (`AGENTS.md`, skills symlinks) | **rarely as a coder** — modify `think/`, `apps/`, or `talent/`, not journal data | `talent/journal/SKILL.md` | +| `journal/` | The live journal (user data). Git-ignored content; checked-in template (`AGENTS.md`, skills symlinks) | **rarely as a coder** — modify `solstone/think/`, `solstone/apps/`, or `solstone/talent/`, not journal data | `solstone/talent/journal/SKILL.md` | Top-level dirs intentionally not in the table: `.venv/`, `scratch/`, `logs/`, `tmp/`, `observers/`, `routines/`, `skills/` — not active coder surfaces. @@ -50,17 +50,17 @@ Top-level dirs intentionally not in the table: `.venv/`, `scratch/`, `logs/`, `t **The pipeline:** `observe` (capture) → JSON transcripts in `journal/chronicle/YYYYMMDD/` → `think` (analyze) → SQLite index + derived artifacts → `convey` (web UI) and `sol call` CLIs. -**Think is the center.** observe feeds it raw material; convey + apps render its outputs; talent prompts + cortex run AI against it; indexer makes it searchable. A change in `think/` usually ripples outward. +**Think is the center.** observe feeds it raw material; convey + apps render its outputs; talent prompts + cortex run AI against it; indexer makes it searchable. A change in `solstone/think/` usually ripples outward. **Key concepts, priority-ordered:** -- **Journal** — the on-disk record rooted at `journal/` in the repo. Every day is a `journal/chronicle/YYYYMMDD/` directory. Segments (timestamped capture windows) are anchored to creation/modification time, not content "about" time. `get_journal()` from `think.utils` is the single source of truth for journal path resolution; trust it unconditionally. Installed runs inherit `SOLSTONE_JOURNAL` from the managed wrapper at `~/.local/bin/sol`; tests use the autouse fixture; sandboxes set it explicitly. Application code must not set it itself (see §8). -- **Talents** — AI processors (markdown prompt + optional Python post-hook). Each has a config in `talent/.md` with frontmatter that declares hooks, priority, model, and output. Cortex spawns them as subprocesses. +- **Journal** — the on-disk record rooted at `journal/` in the repo. Every day is a `journal/chronicle/YYYYMMDD/` directory. Segments (timestamped capture windows) are anchored to creation/modification time, not content "about" time. `get_journal()` from `solstone.think.utils` is the single source of truth for journal path resolution; trust it unconditionally. Installed runs inherit `SOLSTONE_JOURNAL` from the managed wrapper at `~/.local/bin/sol`; tests use the autouse fixture; sandboxes set it explicitly. Application code must not set it itself (see §8). +- **Talents** — AI processors (markdown prompt + optional Python post-hook). Each has a config in `solstone/talent/.md` with frontmatter that declares hooks, priority, model, and output. Cortex spawns them as subprocesses. - **Callosum** — Unix-socket JSON message bus at `journal/health/callosum.sock`. Real-time event distribution across services (`tract` + `event` + payload). If components need to talk asynchronously, they talk through callosum. -- **Cortex** — process manager for talent runs. Listens on callosum (`tract="cortex"`, `event="request"`), spawns `python -m think.talents` subprocesses, writes `/_active.jsonl` then renames to `/.jsonl` on completion, broadcasts all events back through callosum. Read `docs/CORTEX.md` before modifying talent execution. +- **Cortex** — process manager for talent runs. Listens on callosum (`tract="cortex"`, `event="request"`), spawns `python -m solstone.think.talents` subprocesses, writes `/_active.jsonl` then renames to `/.jsonl` on completion, broadcasts all events back through callosum. Read `docs/CORTEX.md` before modifying talent execution. - **Facets** — project/context scopes (`work`, `personal`, …). Group related entities, activities, and relationships. Facet data lives under `journal/facets//`. - **Entities** — tracked people / projects / tools. Extracted from transcripts and accumulated across time. Canonical records in `journal/entities//entity.json`. -- **Activities** — scheduled or observed "things that happen" (meetings, deadlines, anticipated events). Per-facet JSONL at `journal/facets//activities/.jsonl`. Sources: `anticipated` (from `talent/schedule.md`), `user` (manual), `cogitate` (talent-inferred). +- **Activities** — scheduled or observed "things that happen" (meetings, deadlines, anticipated events). Per-facet JSONL at `journal/facets//activities/.jsonl`. Sources: `anticipated` (from `solstone/talent/schedule.md`), `user` (manual), `cogitate` (talent-inferred). - **Indexer** — reads journal state, builds SQLite + FTS5 index. **Never** mutates source data (§7 L6). Rerunning on unchanged data is a no-op. - **Supervisor** — top-level process manager. Starts/restarts services, talks to callosum. `sol supervisor` / `sol start`. @@ -68,13 +68,13 @@ Top-level dirs intentionally not in the table: `.venv/`, `scratch/`, `logs/`, `t Two surfaces: -- **`sol `** — top-level commands registered in `think/sol_cli.py`'s `COMMANDS` dict (e.g., `sol import`, `sol think`, `sol indexer`, `sol supervisor`, `sol heartbeat`). `ALIASES` provides a couple of shorthand compound commands (`sol start` → `sol supervisor`, `sol up/down` → `sol service up/down`). -- **`sol call `** — routes to `think/call.py`, which discovers each `apps/*/call.py` Typer sub-app and mounts it as a subcommand. Example: `sol call entities list`, `sol call activities create`, `sol call journal search`. +- **`sol `** — top-level commands registered in `solstone/think/sol_cli.py`'s `COMMANDS` dict (e.g., `sol import`, `sol think`, `sol indexer`, `sol supervisor`, `sol heartbeat`). `ALIASES` provides a couple of shorthand compound commands (`sol start` → `sol supervisor`, `sol up/down` → `sol service up/down`). +- **`sol call `** — routes to `solstone/think/call.py`, which discovers each `solstone/apps/*/call.py` Typer sub-app and mounts it as a subcommand. Example: `sol call entities list`, `sol call activities create`, `sol call journal search`. -**Adding a top-level command:** add an entry to `COMMANDS` in `think/sol_cli.py`; ensure the module has a `main()` function. +**Adding a top-level command:** add an entry to `COMMANDS` in `solstone/think/sol_cli.py`; ensure the module has a `main()` function. -**Adding a `sol call` sub-verb:** add it to the app's `apps//call.py` Typer sub-app. No central registration needed — `think/call.py` discovers apps automatically. -`sol call journal export` is the CLI entry for portable journal ZIPs; read-only archive validation lives in `think/importers/journal_archive.py`. +**Adding a `sol call` sub-verb:** add it to the app's `solstone/apps//call.py` Typer sub-app. No central registration needed — `solstone/think/call.py` discovers apps automatically. +`sol call journal export` is the CLI entry for portable journal ZIPs; read-only archive validation lives in `solstone/think/importers/journal_archive.py`. Run `sol` (no args) for live status plus the full grouped command list. @@ -87,7 +87,7 @@ Verified against `Makefile`. Grouped by use. | Target | When to use | |--------|-------------| | `make install` | First setup and whenever `pyproject.toml` or `uv.lock` changes. Creates `.venv/`, syncs deps, runs `make skills`. | -| `make skills` | After adding or renaming a `SKILL.md` under `talent/` or `apps/*/talent/`. Rewrites the `.claude/` + `.agents/` skill symlinks into `journal/`. (`make install` depends on this; rarely run alone.) | +| `make skills` | After adding or renaming a `SKILL.md` under `solstone/talent/` or `solstone/apps/*/talent/`. Rewrites the `.claude/` + `.agents/` skill symlinks into `journal/`. (`make install` depends on this; rarely run alone.) | | `make update` | Upgrade all deps to latest, regenerate `uv.lock`. Expect test churn. | | `make update-prices` | Refresh genai-prices model-cost data when adding a new provider model or when pricing tests fail. | | `make clean` | Remove build artifacts, caches, and the skill symlinks. Does not touch `.venv/`. | @@ -108,7 +108,7 @@ Verified against `Makefile`. Grouped by use. | `make format` | Auto-fix formatting and imports with ruff. Safe to run anytime; modifies files. | | `make format-check` | Format dry-run. Part of `make ci`; rarely run alone. | | `make test` | Unit tests (`tests/`). Format-check runs first; failures block tests. Fast inner loop. | -| `make test-apps` | Run all `apps/*/tests/` suites. | +| `make test-apps` | Run all `solstone/apps/*/tests/` suites. | | `make test-app APP=` | Run a single app's tests. | | `make test-only TEST=` | Run a specific test file or pytest node id (`TEST="-k test_name"` also works). | | `make test-integration` | Full integration suite. Requires `.env` API keys. Slow; run before shipping AI-behavior changes. | @@ -151,7 +151,7 @@ Verified against `Makefile`. Grouped by use. | Target | Why not | |--------|---------| -| `make uninstall` | Disabled by design. Use `sol service uninstall`, `sol skills uninstall`, and `python -m think.install_guard uninstall` for installed user artifacts, or `make clean-install` to rebuild the local dev env. | +| `make uninstall` | Disabled by design. Use `sol service uninstall`, `sol skills uninstall`, and `python -m solstone.think.install_guard uninstall` for installed user artifacts, or `make clean-install` to rebuild the local dev env. | ## 6. Testing quickstart @@ -160,14 +160,14 @@ Verified against `Makefile`. Grouped by use. - **Run one test:** `make test-only TEST=tests/test_utils.py::test_foo` or `TEST="-k test_foo"`. - **Run app tests:** `make test-apps` or `make test-app APP=`. - **Integration tests** (`tests/integration/`): hit real provider APIs, require `.env` keys, run via `make test-integration`. -- **After editing `convey/` or `apps/`:** `sol restart-convey` to reload code in a running stack. +- **After editing `solstone/convey/` or `solstone/apps/`:** `sol restart-convey` to reload code in a running stack. - **`make dev` + `make sandbox`** both write runtime artifacts into the fixtures journal; `tests/fixtures/journal/.gitignore` covers those — never commit them. Full depth: `docs/testing.md`. ## 7. Layer hygiene — required reading (L1–L9) -**Why this lives here.** A codebase-wide audit in April 2026 found 14 layer-hygiene violations in `think/` and `apps/`. Infrastructure modules (indexer, importers, schedulers) were silently writing domain state; CLI read-verbs were mutating; get-prefixed functions were creating records on miss. These invariants encode the rules the audit distilled, so the same landmines don't get re-planted. They're inlined here because a one-click-away invariant is a routinely-skipped invariant. +**Why this lives here.** A codebase-wide audit in April 2026 found 14 layer-hygiene violations in `solstone/think/` and `solstone/apps/`. Infrastructure modules (indexer, importers, schedulers) were silently writing domain state; CLI read-verbs were mutating; get-prefixed functions were creating records on miss. These invariants encode the rules the audit distilled, so the same landmines don't get re-planted. They're inlined here because a one-click-away invariant is a routinely-skipped invariant. The low-bar grep enforcement is `scripts/check_layer_hygiene.py`, wired into `make ci`. Known audit-flagged files are allowlisted with audit-reference TODOs; the allowlist shrinks as remediation bundles ship. @@ -181,12 +181,12 @@ Each domain has exactly **one** write-owning module (or one tightly-scoped famil | Domain | Write-owning module(s) | |--------|------------------------| -| Entities (`entities/*/entity.json`, `entities/*/*.npz`) | `think/entities/journal.py` + `think/entities/consolidation.py` + `think/entities/saving.py` + `think/entities/merge.py` + `apps/entities/call.py` | -| Facets (`facets/*/facet.json`, `facets/*/relationships/`) | `think/facets.py` + `apps/facets/*` (if/when created) | -| Observations (`observations.jsonl`) | `think/entities/observations.py` | -| Activities (`facets/*/activities/*.jsonl`) | `think/activities.py` | +| Entities (`entities/*/entity.json`, `entities/*/*.npz`) | `solstone/think/entities/journal.py` + `solstone/think/entities/consolidation.py` + `solstone/think/entities/saving.py` + `solstone/think/entities/merge.py` + `solstone/apps/entities/call.py` | +| Facets (`facets/*/facet.json`, `facets/*/relationships/`) | `solstone/think/facets.py` + `solstone/apps/facets/*` (if/when created) | +| Observations (`observations.jsonl`) | `solstone/think/entities/observations.py` | +| Activities (`facets/*/activities/*.jsonl`) | `solstone/think/activities.py` | | Chronicle day content (`chronicle/YYYYMMDD/**`) | The capturing module (observer, importer) per its declared outputs | -| Index (SQLite, `indexer/*`) | `think/indexer/*` | +| Index (SQLite, `indexer/*`) | `solstone/think/indexer/*` | If you're about to write to a domain from a module not in this table, stop and route through the owner. @@ -227,11 +227,11 @@ An indexer's job is to build indexes from source-of-truth data. Indexers may not ### L7 — Importers only write to imports/ -Importers write source material to `imports/` and the raw-content areas of `chronicle/`. They may not create or modify entities, facets, observations, or other cross-cutting state. If an importer needs to create an entity for deduplication, it calls a domain-owned `seed_entity()` function in `think/entities/` that surfaces the write explicitly. +Importers write source material to `imports/` and the raw-content areas of `chronicle/`. They may not create or modify entities, facets, observations, or other cross-cutting state. If an importer needs to create an entity for deduplication, it calls a domain-owned `seed_entity()` function in `solstone/think/entities/` that surfaces the write explicitly. ### L8 — Hooks have declared outputs -Post-processing hooks (`think/hooks.py`, `talent/*.py` hook functions) declare every path they will write in their frontmatter. The hook runner validates that all actual writes match the declaration. Writes outside the declared set fail loudly — raise at runtime; assert in tests. +Post-processing hooks (`solstone/think/hooks.py`, `solstone/talent/*.py` hook functions) declare every path they will write in their frontmatter. The hook runner validates that all actual writes match the declaration. Writes outside the declared set fail loudly — raise at runtime; assert in tests. ### L9 — Event handlers are idempotent @@ -242,7 +242,7 @@ Any function that handles a callosum event, a scheduled tick, or a supervisor-st The rules above govern *where* code lives. The rules below govern *how* code behaves. They exist because we got burned. - **No backwards-compatibility shims.** All code that depends on this project lives in this repository — never add fallback aliases, re-exports for moved symbols, deprecated-parameter handling, or legacy support code. When renaming or removing something, update every usage directly. For journal data-format changes, write a migration script (see `docs/APPS.md` for `maint` commands); do not add a compatibility layer. Cogitate agents default to adding shims; resist this. -- **Trust `get_journal()` unconditionally.** `get_journal()` from `think.utils` is the single source of truth for journal path resolution. The managed wrapper at `~/.local/bin/sol` sets `SOLSTONE_JOURNAL` for installed runs; tests use the autouse fixture; Makefile sandboxes set it explicitly. Application code, agent prompts, subprocess environments, and service files must not set `SOLSTONE_JOURNAL` themselves. To rewrite the wrapper's embedded path use `sol config journal `. See `docs/environment.md`. +- **Trust `get_journal()` unconditionally.** `get_journal()` from `solstone.think.utils` is the single source of truth for journal path resolution. The managed wrapper at `~/.local/bin/sol` sets `SOLSTONE_JOURNAL` for installed runs; tests use the autouse fixture; Makefile sandboxes set it explicitly. Application code, agent prompts, subprocess environments, and service files must not set `SOLSTONE_JOURNAL` themselves. To rewrite the wrapper's embedded path use `sol config journal `. See `docs/environment.md`. - **SPDX header on every source file.** All Python (and other source) files begin with: ```python @@ -260,7 +260,7 @@ Generic software principles (DRY, KISS, YAGNI, single responsibility, small focu - **SPDX header** as above — mandatory on source code files. - **Naming:** modules / functions / variables `snake_case`; classes `PascalCase`; constants `UPPER_SNAKE_CASE`; private members `_leading_underscore`. Full table in `docs/coding-standards.md`. -- **Imports:** prefer absolute (`from think.utils import get_journal`), grouped stdlib → third-party → local, one per line. +- **Imports:** prefer absolute (`from solstone.think.utils import get_journal`), grouped stdlib → third-party → local, one per line. - **Type hints** on function signatures; `mypy` via `make check`. - **Dependencies:** managed by [uv](https://docs.astral.sh/uv/). `pyproject.toml` is authoritative; `uv.lock` is committed; `make install` syncs; `make update` refreshes. - **Python 3.11+.** @@ -278,7 +278,7 @@ Bare links don't motivate clicking. Each entry below says when you actually need | Doc | When to read | |-----|--------------| -| `docs/APPS.md` | **Required before modifying `apps/`** — pattern catalog for Convey apps, hook-idempotency guidance, Typer sub-app conventions, `maint` commands for data migrations | +| `docs/APPS.md` | **Required before modifying `solstone/apps/`** — pattern catalog for Convey apps, hook-idempotency guidance, Typer sub-app conventions, `maint` commands for data migrations | | `docs/THINK.md` | Understanding the think-layer pipeline (importers, indexer, segment/stream processing) | | `docs/CORTEX.md` | Modifying talent execution, cortex lifecycle, talent process management | | `docs/CALLOSUM.md` | Adding a new tract/event, debugging message flow | @@ -296,9 +296,9 @@ Bare links don't motivate clicking. Each entry below says when you actually need | `docs/INTEGRATION_TESTS.md` | Deep integration-test setup | | `docs/VENDOR.md` | Vendor-level integrations | | `docs/design/` | Per-subsystem design docs | -| `docs/JOURNAL.md` | **Breadcrumb only** — redirects to `talent/journal/SKILL.md`, the progressive-disclosure journal-layout reference | -| `talent/journal/SKILL.md` | Journal layout, vocabulary, and `sol call journal` CLI (loaded by cogitate talents on demand via skills) | -| `talent/journal/references/cli.md` | Full `sol call journal` reference, including **Talent CLI Boundaries** (which infrastructure commands cogitate talents must not call) | +| `docs/JOURNAL.md` | **Breadcrumb only** — redirects to `solstone/talent/journal/SKILL.md`, the progressive-disclosure journal-layout reference | +| `solstone/talent/journal/SKILL.md` | Journal layout, vocabulary, and `sol call journal` CLI (loaded by cogitate talents on demand via skills) | +| `solstone/talent/journal/references/cli.md` | Full `sol call journal` reference, including **Talent CLI Boundaries** (which infrastructure commands cogitate talents must not call) | The live journal also carries `journal/AGENTS.md` as its runtime-facing breadcrumb. @@ -306,8 +306,8 @@ The live journal also carries `journal/AGENTS.md` as its runtime-facing breadcru ## 12. What this file is NOT -- **Not a runtime guide for cogitate talents.** Runtime CLI restrictions on talents live in `talent/journal/references/cli.md` § Talent CLI Boundaries. If you're tuning what a talent can or cannot call, look there, not here. -- **Not the journal-layout reference.** `talent/journal/SKILL.md` + its `references/` is the cogitate-audience entry point. This file describes *how those commands are implemented*, not *which ones talents can't call*. +- **Not a runtime guide for cogitate talents.** Runtime CLI restrictions on talents live in `solstone/talent/journal/references/cli.md` § Talent CLI Boundaries. If you're tuning what a talent can or cannot call, look there, not here. +- **Not the journal-layout reference.** `solstone/talent/journal/SKILL.md` + its `references/` is the cogitate-audience entry point. This file describes *how those commands are implemented*, not *which ones talents can't call*. - **Not an operations manual.** For debugging a live system see `docs/DOCTOR.md`; for setup and service lifecycle, see `docs/INSTALL.md`, `sol setup`, and `sol service`. ## 13. Owner-facing copy: the system-anatomy canon @@ -315,11 +315,11 @@ The live journal also carries `journal/AGENTS.md` as its runtime-facing breadcru - **The trinity.** In owner-facing copy, name the system in canonical order: `solstone = observers + sol agent + journal`. - **The canon lives elsewhere.** The source of truth is sol pbc's internal brand canon (system anatomy + voice terminology guides). This repo's branded prose follows it; the canon itself is not vendored here. - **Ban surveillance verbs in branded surfaces.** Never use "capture", "watch", "record", "monitor", "track", or "collect" in template copy, settings labels, error messages, onboarding text, or README / INSTALL prose. Prefer "observe alongside", "experience along with", or "take in what you take in". -- **`capture` is code-only.** Keep it in module names such as `observe/`, function names, OS subsystem identifiers such as `com.solstone.capture`, and internal architecture diagrams. That is intentional and aligned with the canon. +- **`capture` is code-only.** Keep it in module names such as `solstone/observe/`, function names, OS subsystem identifiers such as `com.solstone.capture`, and internal architecture diagrams. That is intentional and aligned with the canon. - **Name artifacts for owners, not pipelines.** In branded prose, say "raw media", "the originals", or "observations". Never say "raw captures" or "screen captures" in owner-facing strings. Code-side artifact names stay as-is. - **`sol` is one thing.** `sol` is the running software; there is no homunculus behind it. Use two registers for one entity: `sol` in conversation, `sol agent` in technical contexts. - **`keeper` is a surface-specific edge case.** `voice-terminology.md` makes `keeper` the role noun for `sol` in product copy generally. The `solstone-swift` surface bans `keeper` because the mobile UX uses the owner's chosen identity, default `sol`. When writing copy for a specific surface, follow that surface's terminology covenant. -- **Edit with the right mental model.** Internal architecture vocabulary in this repo stays as-is: `observe/`, the capture pipeline, and screen capture log subsystems remain correct code language. Apply the canon to owner-facing strings only: UI copy, settings text, install / README prose, error messages, and onboarding. If an owner sees it, follow the canon; if it's code or internal docs about pipelines, `capture` is fine. +- **Edit with the right mental model.** Internal architecture vocabulary in this repo stays as-is: `solstone/observe/`, the capture pipeline, and screen capture log subsystems remain correct code language. Apply the canon to owner-facing strings only: UI copy, settings text, install / README prose, error messages, and onboarding. If an owner sees it, follow the canon; if it's code or internal docs about pipelines, `capture` is fine. | Surface | Terminology rule | |---------|------------------| diff --git a/INSTALL.md b/INSTALL.md index 8ae1036bb..65d3ad00c 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -72,7 +72,7 @@ brew install git uv - **macOS (Apple Silicon):** Xcode command line tools are required because the helper is a Swift package; if the `xcodebuild -version` check above fails, fix that first. `sol install-models` downloads roughly 461 MB of model data into `~/Library/Application Support/solstone/parakeet/models`. - **Linux (x86_64):** `make install` auto-detects `PARAKEET_ONNX_VARIANT` (`cuda` when `nvidia-smi -L` succeeds, otherwise `cpu`), runs `uv sync --extra parakeet-onnx-`, then runs `sol install-models`. Override direct model installation with `sol install-models --variant cuda` or `sol install-models --variant cpu`; `PARAKEET_ONNX_VARIANT` is honored for one release cycle as a fallback when `--variant auto` is used. The CPU extra installs `parakeet-onnx-cpu`; the CUDA extra installs `parakeet-onnx-cuda`. The ONNX model footprint is roughly 2.55 GB, and the CUDA wheels add roughly 700 MB more when using the CUDA variant. - To use Whisper instead, set `transcribe.backend = "whisper"` in `journal/config/journal.json` or switch the backend in the settings UI. The Whisper code and dependencies remain available for rollback. -- helper contract details live in `observe/transcribe/parakeet_helper/README.md`. +- helper contract details live in `solstone/observe/transcribe/parakeet_helper/README.md`. ## install diff --git a/Makefile b/Makefile index 067ab4ed6..1acbb9fd3 100644 --- a/Makefile +++ b/Makefile @@ -72,10 +72,10 @@ uv.lock: pyproject.toml # Install package in editable mode with isolated venv install: .installed - @(cd /tmp && $(CURDIR)/$(VENV_BIN)/python -c "from think.sol_cli import main") 2>/dev/null || { \ + @(cd /tmp && $(CURDIR)/$(VENV_BIN)/python -c "from solstone.think.sol_cli import main") 2>/dev/null || { \ echo ">>> re-registering editable install"; \ $(UV) pip install -e . --no-deps; \ - if (cd /tmp && $(CURDIR)/$(VENV_BIN)/python -c "from think.sol_cli import main"); then \ + if (cd /tmp && $(CURDIR)/$(VENV_BIN)/python -c "from solstone.think.sol_cli import main"); then \ echo ">>> re-registered successfully"; \ else \ echo ">>> editable install still broken; run make clean-install"; \ @@ -236,12 +236,12 @@ install-models: # Build the parakeet helper binary (macOS/arm64 only, requires Xcode CLT) parakeet-helper: - cd observe/transcribe/parakeet_helper && swift build -c release - @echo "built: $$(pwd)/observe/transcribe/parakeet_helper/.build/release/parakeet-helper" + cd solstone/observe/transcribe/parakeet_helper && swift build -c release + @echo "built: $$(pwd)/solstone/observe/transcribe/parakeet_helper/.build/release/parakeet-helper" # Remove parakeet helper build artifacts parakeet-helper-clean: - rm -rf observe/transcribe/parakeet_helper/.build observe/transcribe/parakeet_helper/.swiftpm observe/transcribe/parakeet_helper/Package.resolved + rm -rf solstone/observe/transcribe/parakeet_helper/.build solstone/observe/transcribe/parakeet_helper/.swiftpm solstone/observe/transcribe/parakeet_helper/Package.resolved # Run browser scenarios against sandbox verify-browser: .installed @@ -330,7 +330,7 @@ test: .installed format-check # Run app tests test-apps: .installed @echo "Running app tests..." - $(PYTEST_BASETEMP_INIT) $(TEST_ENV) $(PYTEST) $(PYTEST_BASETEMP_FLAG) apps/ -q + $(PYTEST_BASETEMP_INIT) $(TEST_ENV) $(PYTEST) $(PYTEST_BASETEMP_FLAG) solstone/apps/ -q # Run specific app tests test-app: .installed @@ -339,7 +339,7 @@ test-app: .installed echo "Example: make test-app APP=todos"; \ exit 1; \ fi - $(PYTEST_BASETEMP_INIT) $(TEST_ENV) $(PYTEST) $(PYTEST_BASETEMP_FLAG) apps/$(APP)/tests/ -v + $(PYTEST_BASETEMP_INIT) $(TEST_ENV) $(PYTEST) $(PYTEST_BASETEMP_FLAG) solstone/apps/$(APP)/tests/ -v # Run specific test file or pattern test-only: .installed @@ -377,7 +377,7 @@ test-integration-only: .installed # Run all tests (core + apps + integration) test-all: .installed @echo "Running all tests (core + apps + integration)..." - $(PYTEST_BASETEMP_INIT) $(TEST_ENV) $(PYTEST) $(PYTEST_BASETEMP_FLAG) tests/ -v --cov=. --ignore=tests/integration $(LINK_LIVE_TESTS) && $(TEST_ENV) $(PYTEST) $(PYTEST_BASETEMP_FLAG) apps/ -v --cov=. --cov-append + $(PYTEST_BASETEMP_INIT) $(TEST_ENV) $(PYTEST) $(PYTEST_BASETEMP_FLAG) tests/ -v --cov=. --ignore=tests/integration $(LINK_LIVE_TESTS) && $(TEST_ENV) $(PYTEST) $(PYTEST_BASETEMP_FLAG) solstone/apps/ -v --cov=. --cov-append # Auto-format and fix code, then report any remaining issues format: .installed @@ -414,7 +414,7 @@ service-logs: $(VENV_BIN)/sol service logs -f uninstall: - @echo "Error: 'make uninstall' is disabled. Use 'sol service uninstall', 'sol skills uninstall', and 'python -m think.install_guard uninstall' to remove installed user artifacts, or 'make clean-install' to rebuild the local dev environment." >&2 + @echo "Error: 'make uninstall' is disabled. Use 'sol service uninstall', 'sol skills uninstall', and 'python -m solstone.think.install_guard uninstall' to remove installed user artifacts, or 'make clean-install' to rebuild the local dev environment." >&2 @exit 1 FORCE: @@ -459,7 +459,7 @@ watch: .installed # Generate coverage report (core + apps, excluding core integration tests) coverage: .installed $(PYTEST_BASETEMP_INIT) $(TEST_ENV) $(PYTEST) $(PYTEST_BASETEMP_FLAG) tests/ --cov=. --cov-report=html --cov-report=term --ignore=tests/integration $(LINK_LIVE_TESTS) - $(PYTEST_BASETEMP_INIT) $(TEST_ENV) $(PYTEST) $(PYTEST_BASETEMP_FLAG) apps/ --cov=. --cov-report=html --cov-report=term --cov-append + $(PYTEST_BASETEMP_INIT) $(TEST_ENV) $(PYTEST) $(PYTEST_BASETEMP_FLAG) solstone/apps/ --cov=. --cov-report=html --cov-report=term --cov-append @echo "Coverage report generated in htmlcov/index.html" # Update all dependencies to latest versions and refresh genai-prices diff --git a/README.md b/README.md index 5bd620010..d3f4a0aad 100644 --- a/README.md +++ b/README.md @@ -60,7 +60,7 @@ Python 3.11+, Linux + macOS, AGPL-3.0-only, maintained by [sol pbc](https://solp ``` - **observe** — receives audio and screen observations from standalone observers (solstone-linux, solstone-tmux, solstone-macos) via observer ingest. processes FLAC audio, WebM screen media, and timestamped metadata. -- **think** — transcribes audio (faster-whisper), analyzes screen observations, extracts entities, detects meetings, and indexes everything into SQLite. runs 30 configurable agent/generator templates from `talent/`. +- **think** — transcribes audio (faster-whisper), analyzes screen observations, extracts entities, detects meetings, and indexes everything into SQLite. runs 30 configurable agent/generator templates from `solstone/talent/`. - **cortex** — orchestrates agent execution. receives events, dispatches agents, writes results back to the journal. - **callosum** — async message bus connecting all services. enables event-driven coordination between observe, think, cortex, and convey. - **convey** — Flask-based web interface with 17 pluggable apps for navigating journal data. diff --git a/apps/README.md b/apps/README.md deleted file mode 120000 index b82e687b6..000000000 --- a/apps/README.md +++ /dev/null @@ -1 +0,0 @@ -../docs/APPS.md \ No newline at end of file diff --git a/convey/README.md b/convey/README.md deleted file mode 120000 index 68e3811fa..000000000 --- a/convey/README.md +++ /dev/null @@ -1 +0,0 @@ -../docs/CONVEY.md \ No newline at end of file diff --git a/convey/static/tests/README.md b/convey/static/tests/README.md deleted file mode 100644 index 84f627c72..000000000 --- a/convey/static/tests/README.md +++ /dev/null @@ -1,5 +0,0 @@ -Open `convey/static/tests/api.html` in a browser; each assertion reports pass/fail inline. -Open `convey/static/tests/relative-time.html` in a browser; each assertion reports pass/fail inline. -Open `convey/static/tests/surface-state.html` in a browser; each assertion reports pass/fail inline. -Open `convey/static/tests/ws-listen.html` in a browser; each assertion reports pass/fail inline. -Open `convey/static/tests/register-task.html` in a browser; each assertion reports pass/fail inline. diff --git a/docs/APPS.md b/docs/APPS.md index 1d9623fde..c807a29f3 100644 --- a/docs/APPS.md +++ b/docs/APPS.md @@ -1,6 +1,6 @@ # solstone App Development Guide -**Complete guide for building apps in the `apps/` directory.** +**Complete guide for building apps in the `solstone/apps/` directory.** Apps are the primary way to extend solstone's web interface (Convey). Each app is a self-contained module discovered automatically using **convention over configuration**—no base classes or manual registration required. @@ -14,10 +14,10 @@ Create a minimal app in two steps: ```bash # 1. Create app directory (use underscores, not hyphens!) -mkdir apps/my_app +mkdir solstone/apps/my_app # 2. Create workspace template -touch apps/my_app/workspace.html +touch solstone/apps/my_app/workspace.html ``` **Minimal `workspace.html`:** @@ -61,7 +61,7 @@ apps/my_app/ | `app.json` | No | Icon, label, facet support overrides | | `app_bar.html` | No | Bottom fixed bar for app controls | | `background.html` | No | Background service (WebSocket listeners) | -| `talent/` | No | Custom agents, generators, and skills (`.md` files + skill subdirectories) | +| `solstone/talent/` | No | Custom agents, generators, and skills (`.md` files + skill subdirectories) | | `maint/` | No | One-time maintenance tasks (run on Convey startup) | | `tests/` | No | App-specific tests with self-contained fixtures | @@ -78,7 +78,7 @@ apps/my_app/ **Index route**: All apps are automatically served at `/app/{app_name}` via a shared handler. You don't need to define an index route in `routes.py`. -See `apps/__init__.py` for discovery logic and route injection. +See `solstone/apps/__init__.py` for discovery logic and route injection. --- @@ -104,9 +104,9 @@ The workspace template is included inside the app container (`app.html`). - See [VENDOR.md](VENDOR.md) for available libraries **Reference implementations:** -- Minimal: `apps/home/workspace.html` (simple content) -- Styled: `apps/support/workspace.html` (custom CSS, forms, interactive JS) -- Data-driven: `apps/todos/workspace.html` (facet sections, dynamic rendering) +- Minimal: `solstone/apps/home/workspace.html` (simple content) +- Styled: `solstone/apps/support/workspace.html` (custom CSS, forms, interactive JS) +- Data-driven: `solstone/apps/todos/workspace.html` (facet sections, dynamic rendering) --- @@ -126,10 +126,10 @@ Define custom routes for your app (API endpoints, form handlers, navigation rout - Import utilities from `convey.utils` (see [Flask Utilities](#flask-utilities)) **Reference implementations:** -- API endpoints: `apps/search/routes.py` (search APIs, no index route) -- Form handlers: `apps/todos/routes.py` (POST handlers, validation, flash messages) -- Navigation: `apps/activities/routes.py` (date-based routes with custom context) -- Redirects: `apps/todos/routes.py` index route (redirects `/` to today's date) +- API endpoints: `solstone/apps/search/routes.py` (search APIs, no index route) +- Form handlers: `solstone/apps/todos/routes.py` (POST handlers, validation, flash messages) +- Navigation: `solstone/apps/activities/routes.py` (date-based routes with custom context) +- Redirects: `solstone/apps/todos/routes.py` index route (redirects `/` to today's date) @@ -137,7 +137,7 @@ Define custom routes for your app (API endpoints, form handlers, navigation rout Override default icon, label, and other app settings. -**Authoritative source:** See the `App` dataclass in `apps/__init__.py` for all supported fields, types, and defaults. +**Authoritative source:** See the `App` dataclass in `solstone/apps/__init__.py` for all supported fields, types, and defaults. **Common fields:** - `icon` - Emoji icon for menu bar (default: "📦") @@ -148,7 +148,7 @@ Override default icon, label, and other app settings. **When to disable facets:** Set `"facets": false` for apps that don't use facet-based organization (e.g., system settings, dev tools). -**Examples:** Browse `apps/*/app.json` for reference configurations. +**Examples:** Browse `solstone/apps/*/app.json` for reference configurations. ### 4. `app_bar.html` - Bottom Bar Controls @@ -221,9 +221,9 @@ Both badge types appear as red notification counts. - See [CALLOSUM.md](CALLOSUM.md) for event protocol details **Reference implementations:** -- `apps/todos/background.html` - App icon badge with API fetch +- `solstone/apps/todos/background.html` - App icon badge with API fetch -**Implementation source:** `convey/static/app.js` - AppServices framework, `convey/static/websocket.js` - WebSocket API +**Implementation source:** `solstone/convey/static/app.js` - AppServices framework, `solstone/convey/static/websocket.js` - WebSocket API --- @@ -237,8 +237,8 @@ Define plain callable tool functions for your app in `tools.py`. - Put shared logic in your app/module layer and call it from these functions **Reference implementations:** -- `apps/todos/tools.py` -- `apps/entities/tools.py` +- `solstone/apps/todos/tools.py` +- `solstone/apps/entities/tools.py` --- @@ -267,27 +267,27 @@ app = typer.Typer(help="Description of your app commands.") - Print formatted text instead of returning dicts - Use `typer.Exit(1)` for errors instead of returning error dicts -**Discovery behavior:** The `sol call` dispatcher scans `apps/*/call.py` at startup, imports modules, and mounts any `app` variable that is a `typer.Typer` instance as a sub-command. Private apps (directories starting with `_`) are skipped. +**Discovery behavior:** The `sol call` dispatcher scans `solstone/apps/*/call.py` at startup, imports modules, and mounts any `app` variable that is a `typer.Typer` instance as a sub-command. Private apps (directories starting with `_`) are skipped. **Reference implementations:** -- Discovery logic: `think/call.py` - `_discover_app_calls()` function -- App CLI example: `apps/todos/call.py` - Todo list command +- Discovery logic: `solstone/think/call.py` - `_discover_app_calls()` function +- App CLI example: `solstone/apps/todos/call.py` - Todo list command -**Skills app reference:** `apps/skills/call.py` is the current owner-wide pattern for a data-backed app CLI. It exposes `sol call skills list|show|observe|seed|promote|refresh|mark-dormant|retire|edit-request|rename` and routes all writes through `think/skills.py`, which owns `journal/skills/patterns.jsonl`, `journal/skills/edit_requests.jsonl`, and `journal/skills/{slug}.md`. The shipped daily talents for this app live in `apps/skills/talent/skill_observer.md` (daily cogitate, priority 41) and `apps/skills/talent/skill_editor.md` + `skill_editor.py` (daily generate, priority 60). The observer marks patterns for creation/refresh, and the editor consumes those flags or pending `edit-request` rows to write/update exactly one owner-wide profile per run. +**Skills app reference:** `solstone/apps/skills/call.py` is the current owner-wide pattern for a data-backed app CLI. It exposes `sol call skills list|show|observe|seed|promote|refresh|mark-dormant|retire|edit-request|rename` and routes all writes through `solstone/think/skills.py`, which owns `journal/skills/patterns.jsonl`, `journal/skills/edit_requests.jsonl`, and `journal/skills/{slug}.md`. The shipped daily talents for this app live in `solstone/apps/skills/talent/skill_observer.md` (daily cogitate, priority 41) and `solstone/apps/skills/talent/skill_editor.md` + `skill_editor.py` (daily generate, priority 60). The observer marks patterns for creation/refresh, and the editor consumes those flags or pending `edit-request` rows to write/update exactly one owner-wide profile per run. --- -### 8. `talent/` - App Generators +### 8. `solstone/talent/` - App Generators Define custom generator prompts that integrate with solstone's output generation system. **Key Points:** -- Create `talent/` directory with `.md` files containing JSON frontmatter +- Create `solstone/talent/` directory with `.md` files containing JSON frontmatter - App generators are automatically discovered alongside system generators - Keys are namespaced as `{app}:{agent}` (e.g., `my_app:weekly_summary`) - Outputs go to `JOURNAL/YYYYMMDD/talents/__.md` (or `.json` if `output: "json"`) -**Metadata format:** Same schema as system generators in `talent/*.md` - JSON frontmatter includes `title`, `description`, `color`, `schedule` (required), `priority` (required for scheduled prompts), `hook`, `output`, `max_output_tokens`, and `thinking_budget` fields. The `schedule` field must be `"segment"` or `"daily"`. The `priority` field is required for all scheduled prompts - prompts without explicit priority will fail validation. Set `output: "json"` for structured JSON output instead of markdown. Optional `max_output_tokens` sets the maximum response length; `thinking_budget` sets the model's thinking token budget (provider-specific defaults apply if omitted). Generators reject a `cwd` field entirely; working-directory control is only available for `type: "cogitate"` prompts. +**Metadata format:** Same schema as system generators in `solstone/talent/*.md` - JSON frontmatter includes `title`, `description`, `color`, `schedule` (required), `priority` (required for scheduled prompts), `hook`, `output`, `max_output_tokens`, and `thinking_budget` fields. The `schedule` field must be `"segment"` or `"daily"`. The `priority` field is required for all scheduled prompts - prompts without explicit priority will fail validation. Set `output: "json"` for structured JSON output instead of markdown. Optional `max_output_tokens` sets the maximum response length; `thinking_budget` sets the model's thinking token budget (provider-specific defaults apply if omitted). Generators reject a `cwd` field entirely; working-directory control is only available for `type: "cogitate"` prompts. **Priority bands:** Prompts run in priority order (lowest first). Recommended bands: - 10-30: Generators (content-producing prompts) @@ -320,7 +320,7 @@ Example: - Use `"hook": {"post": "my_hook"}` for post-processing hooks - Use both together: `"hook": {"pre": "prep", "post": "process"}` - Use `"hook": {"flush": true}` to opt into segment flush (see below) -- Resolution: `"name"` → `talent/{name}.py`, `"app:name"` → `apps/{app}/talent/{name}.py`, or explicit path +- Resolution: `"name"` → `solstone/talent/{name}.py`, `"app:name"` → `solstone/apps/{app}/talent/{name}.py`, or explicit path **Pre-hooks** (`pre_process`): Modify inputs before the LLM call - `context` is the full config dict with: `name`, `use_id`, `provider`, `model`, `prompt`, `system_instruction` (if set), `user_instruction`, `output`, `meta`, and for generators: `day`, `segment`, `span`, `span_mode`, `transcript`, `output_path` @@ -347,42 +347,42 @@ def post_process(result: str, context: dict) -> str | None: return result + "\n\n## Generated by hook" ``` -**Hook idempotency:** Post-hooks that write to shared journal state must be safe to run more than once on the same inputs. `sol think --refresh` bypasses the "output already exists" early-return in `think/talents.py` and re-executes the talent, which re-fires `post_process` against a fresh LLM result — so any side-effect the hook performs (writing events, appending to a log, updating an index file) will happen again. Pick one of these two patterns: +**Hook idempotency:** Post-hooks that write to shared journal state must be safe to run more than once on the same inputs. `sol think --refresh` bypasses the "output already exists" early-return in `solstone/think/talents.py` and re-executes the talent, which re-fires `post_process` against a fresh LLM result — so any side-effect the hook performs (writing events, appending to a log, updating an index file) will happen again. Pick one of these two patterns: - **Natural-key dedup.** Read the existing output, compute a natural key per row (e.g., `(facet, event_day, title, start, end)` for facet events), skip rows already present, and append only the new ones. Use this when the output is append-only history and you want to preserve prior writes from other agents. -- **Atomic replace.** Recompute the full output, write it to a temp file, and rename into place. `atomic_write()` in `think/entities/core.py` is the established helper for text outputs; for JSONL, write the full set of lines to a tempfile and `os.replace()`. Use this when the hook owns the file end-to-end. +- **Atomic replace.** Recompute the full output, write it to a temp file, and rename into place. `atomic_write()` in `solstone/think/entities/core.py` is the established helper for text outputs; for JSONL, write the full set of lines to a tempfile and `os.replace()`. Use this when the hook owns the file end-to-end. -(Retired 2026-04-18 Sprint 4.) An earlier `write_events_jsonl` hook in `think/hooks.py` opened facet-event logs in `"a"` mode with no dedup and doubled row counts on every `sol think --refresh` — see the 2026-04-17 layer-violations audit (V6) tracked in sol pbc's internal engineering notes for the full write-up. +(Retired 2026-04-18 Sprint 4.) An earlier `write_events_jsonl` hook in `solstone/think/hooks.py` opened facet-event logs in `"a"` mode with no dedup and doubled row counts on every `sol think --refresh` — see the 2026-04-17 layer-violations audit (V6) tracked in sol pbc's internal engineering notes for the full write-up. See `docs/coding-standards.md` L8/L9 for the broader principles. **Reference implementations:** -- System generator templates: `talent/*.md` (files with `schedule` field but no `tools` field) -- Schedule hook: `talent/schedule.py` -- Discovery logic: `think/talent.py` - `get_talent_configs(has_tools=False)`, `get_output_name()` -- Hook loading: `think/talent.py` - `load_pre_hook()`, `load_post_hook()` +- System generator templates: `solstone/talent/*.md` (files with `schedule` field but no `tools` field) +- Schedule hook: `solstone/talent/schedule.py` +- Discovery logic: `solstone/think/talent.py` - `get_talent_configs(has_tools=False)`, `get_output_name()` +- Hook loading: `solstone/think/talent.py` - `load_pre_hook()`, `load_post_hook()` --- -### 9. `talent/` - App Agents and Generators +### 9. `solstone/talent/` - App Agents and Generators Define custom agents and generator templates that integrate with solstone's Cortex agent system. **Key Points:** -- Create `talent/` directory with `.md` files containing JSON frontmatter +- Create `solstone/talent/` directory with `.md` files containing JSON frontmatter - Both agents and generators live in the same directory - distinguished by frontmatter fields - Agents have a `tools` field, generators have `schedule` but no `tools` - App agents/generators are automatically discovered alongside system ones - Keys are namespaced as `{app}:{name}` (e.g., `my_app:helper`) - Agents inherit all system agent capabilities (tools, scheduling, multi-facet) -**Metadata format:** Same schema as system agents in `talent/*.md` - JSON frontmatter includes `title`, `provider`, `model`, `tools`, `schedule`, `priority`, `multi_facet`, `max_output_tokens`, and `thinking_budget` fields. The `priority` field is **required** for all scheduled prompts - prompts without explicit priority will fail validation. See the priority bands documentation in [THINK.md](THINK.md#unified-priority-execution). Optional `max_output_tokens` sets the maximum response length; `thinking_budget` sets the model's thinking token budget (provider-specific defaults apply if omitted; OpenAI uses fixed reasoning and ignores this field). Cogitate agents may also declare `cwd: "journal"` or `cwd: "repo"`; when omitted they default to `journal`, and repo-oriented prompts like `coder` should opt into `repo`. See [CORTEX.md](CORTEX.md) for agent configuration details. +**Metadata format:** Same schema as system agents in `solstone/talent/*.md` - JSON frontmatter includes `title`, `provider`, `model`, `tools`, `schedule`, `priority`, `multi_facet`, `max_output_tokens`, and `thinking_budget` fields. The `priority` field is **required** for all scheduled prompts - prompts without explicit priority will fail validation. See the priority bands documentation in [THINK.md](THINK.md#unified-priority-execution). Optional `max_output_tokens` sets the maximum response length; `thinking_budget` sets the model's thinking token budget (provider-specific defaults apply if omitted; OpenAI uses fixed reasoning and ignores this field). Cogitate agents may also declare `cwd: "journal"` or `cwd: "repo"`; when omitted they default to `journal`, and repo-oriented prompts like `coder` should opt into `repo`. See [CORTEX.md](CORTEX.md) for agent configuration details. **Template variables:** Agent prompts can use template variables like `$name`, `$preferred`, and pronoun variables. See [PROMPT_TEMPLATES.md](PROMPT_TEMPLATES.md) for the complete template system documentation. **Reference implementations:** -- System agent examples: `talent/*.md` (files with `tools` field) -- Discovery logic: `think/talent.py` - `get_talent_configs(has_tools=True)`, `get_talent()` +- System agent examples: `solstone/talent/*.md` (files with `tools` field) +- Discovery logic: `solstone/think/talent.py` - `get_talent_configs(has_tools=True)`, `get_talent()` #### Prompt Context Configuration @@ -405,18 +405,18 @@ Context is provided inline in the `.md` body via template variables: - `$facets` - focused facet context or all available facets - `$activity_context` - activity metadata, segment state, and analysis focus sections -**Authoritative source:** `think/talent.py` - `_DEFAULT_LOAD`, `source_is_enabled()`, `source_is_required()`, `get_talent_filter()` +**Authoritative source:** `solstone/think/talent.py` - `_DEFAULT_LOAD`, `source_is_enabled()`, `source_is_required()`, `get_talent_filter()` --- -### 10. `talent/` - Agent Skills +### 10. `solstone/talent/` - Agent Skills -Define [Agent Skills](https://agentskills.io/specification) as subdirectories within `talent/`. Skills package procedural knowledge, workflows, and resources that AI coding agents (Claude Code, GitHub Copilot, Gemini CLI, etc.) can discover and use on demand. +Define [Agent Skills](https://agentskills.io/specification) as subdirectories within `solstone/talent/`. Skills package procedural knowledge, workflows, and resources that AI coding agents (Claude Code, GitHub Copilot, Gemini CLI, etc.) can discover and use on demand. **Key Points:** -- Create a subdirectory in `talent/` with a `SKILL.md` file (YAML frontmatter + markdown body) +- Create a subdirectory in `solstone/talent/` with a `SKILL.md` file (YAML frontmatter + markdown body) - The directory name must match the `name` field in the YAML frontmatter -- Skill names must be unique across system `talent/` and all `apps/*/talent/` directories +- Skill names must be unique across system `solstone/talent/` and all `solstone/apps/*/talent/` directories - `make skills` discovers all skills and symlinks them into `journal/.agents/skills/` and `journal/.claude/skills/` - Skills are standalone — they don't interact with the talent agent/generator system - The talent loader ignores subdirectories, so skills won't interfere with agent discovery @@ -452,14 +452,14 @@ Step-by-step procedures, examples, and domain knowledge for the agent. - `metadata` — Arbitrary key-value string map - `allowed-tools` — Space-delimited list of pre-approved tools (experimental) -**App skills** work the same way — place a skill directory inside `apps/my_app/talent/`: +**App skills** work the same way — place a skill directory inside `solstone/apps/my_app/talent/`: ``` apps/my_app/talent/my-skill/ ├── SKILL.md └── references/ ``` -**Running `make skills`:** Discovers all `SKILL.md` files under `talent/*/` and `apps/*/talent/*/`, then creates symlinks in `journal/.agents/skills/` and `journal/.claude/skills/` so that all supported coding agents see the same skills. Errors if two skills share the same directory name. +**Running `make skills`:** Discovers all `SKILL.md` files under `solstone/talent/*/` and `solstone/apps/*/talent/*/`, then creates symlinks in `journal/.agents/skills/` and `journal/.claude/skills/` so that all supported coding agents see the same skills. Errors if two skills share the same directory name. --- @@ -477,8 +477,8 @@ Define one-time maintenance scripts that run automatically when supervisor start **CLI:** `sol maint` (run pending), `sol maint --list` (show status), `sol maint --force` (re-run all) **Reference implementations:** -- Example task: `apps/entities/maint/001_migrate_to_journal_entities.py` - real migration task demonstrating maint patterns -- Discovery logic: `think/maint.py` - `discover_tasks()`, `run_task()` +- Example task: `solstone/apps/entities/maint/001_migrate_to_journal_entities.py` - real migration task demonstrating maint patterns +- Discovery logic: `solstone/think/maint.py` - `discover_tasks()`, `run_task()` --- @@ -501,8 +501,8 @@ apps/my_app/tests/ ``` **Reference implementations:** -- Fixture patterns: `apps/todos/tests/conftest.py` -- Tool testing: `apps/todos/tests/test_tools.py` +- Fixture patterns: `solstone/apps/todos/tests/conftest.py` +- Tool testing: `solstone/apps/todos/tests/test_tools.py` --- @@ -520,22 +520,22 @@ Define server-side handlers that react to Callosum events. Handlers run in Conve **Available imports** (same as route handlers): - `from convey import state` - Access `state.journal_root` - `from convey import emit` - Emit events back to Callosum -- `from apps.utils import get_app_storage_path, log_app_action` - App storage -- `from convey.utils import load_json, save_json, spawn_agent` - Utilities +- `from solstone.apps.utils import get_app_storage_path, log_app_action` - App storage +- `from solstone.convey.utils import load_json, save_json, spawn_agent` - Utilities **Not available** (no Flask request context): - `request`, `session`, `current_app` - `error_response()`, `success_response()`, `parse_pagination_params()` **Reference implementations:** -- Framework: `apps/events.py` - `EventContext` dataclass, decorator, discovery -- Example: `apps/entities/events.py` - Entity activity tracking via event handlers +- Framework: `solstone/apps/events.py` - `EventContext` dataclass, decorator, discovery +- Example: `solstone/apps/entities/events.py` - Entity activity tracking via event handlers --- ## Flask Utilities -Available in `convey/utils.py`: +Available in `solstone/convey/utils.py`: ### Route Helpers - `error_response(message, code=400)` - Standard JSON error response @@ -552,28 +552,28 @@ Available in `convey/utils.py`: - `load_json(path)` - Load JSON file with error handling (returns None on error) - `save_json(path, data, indent, add_newline)` - Save JSON with formatting (returns bool) -**See source:** `convey/utils.py` for full signatures and documentation +**See source:** `solstone/convey/utils.py` for full signatures and documentation ### App Storage Apps can persist journal-specific configuration and data in `/apps//`: ```python -from apps.utils import get_app_storage_path, load_app_config, save_app_config +from solstone.apps.utils import get_app_storage_path, load_app_config, save_app_config ``` - `get_app_storage_path(app_name, *sub_dirs, ensure_exists)` - Get Path to app storage directory - `load_app_config(app_name, default)` - Load app config from `config.json` - `save_app_config(app_name, config)` - Save app config to `config.json` -**See source:** `apps/utils.py` for implementation details +**See source:** `solstone/apps/utils.py` for implementation details ### Action Logging Apps that modify owner data should log actions for audit trail purposes: ```python -from apps.utils import log_app_action +from solstone.apps.utils import log_app_action ``` - `log_app_action(app, facet, action, params, day=None)` - Log owner-initiated action @@ -598,15 +598,15 @@ Log after successful mutations, not attempts. Available functions from the `think` module: ### Facets -`think/facets.py`: `get_facets()` - Returns dict of facet configurations +`solstone/think/facets.py`: `get_facets()` - Returns dict of facet configurations ### Todos -`apps/todos/todo.py`: +`solstone/apps/todos/todo.py`: - `get_todos(day, facet)` - Get todo list for day and facet - `TodoChecklist` class - Load and manipulate todo markdown files ### Entities -`think/entities/`: `load_entities(facet)` - Load entities for a facet +`solstone/think/entities/`: `load_entities(facet)` - Load entities for a facet See [talent/journal/SKILL.md](../talent/journal/SKILL.md), [CORTEX.md](CORTEX.md), [CALLOSUM.md](CALLOSUM.md) for subsystem details. @@ -616,7 +616,7 @@ See [talent/journal/SKILL.md](../talent/journal/SKILL.md), [CORTEX.md](CORTEX.md ### Global Variables -Defined in `convey/templates/app.html`: +Defined in `solstone/convey/templates/app.html`: - `window.facetsData` - Array of facet objects `[{name, title, color, emoji}, ...]` - `window.selectedFacet` - Current facet name or null (see Facet Selection below) - `window.appFacetCounts` - Badge counts for current app `{"work": 5, "personal": 3}` (set via route's `facet_counts`) @@ -636,13 +636,13 @@ Apps can access and control facet selection through a uniform API: **UX Tip:** Apps should provide visual indication when in all-facet mode vs showing a specific facet. For example, group items by facet, show facet badges/colors on items, or display a subtle "All facets" label. This helps owners understand the scope of what they're viewing. -**See implementation:** `convey/static/app.js` - Facet switching logic and event dispatch +**See implementation:** `solstone/convey/static/app.js` - Facet switching logic and event dispatch **Disabled mode:** On apps with `facets.disabled: true`, the facet bar is visible but inert — pills render without interactivity or tab stops. The container is marked `aria-hidden="true"` so screen readers skip it. The bar remains visually present as always-visible chrome. ### WebSocket Events (Client-Side) -`window.appEvents` API defined in `convey/static/websocket.js`: +`window.appEvents` API defined in `solstone/convey/static/websocket.js`: - `listen(tract, callback)` - Subscribe to specific tract or '*' for all events - Messages structure: `{tract: 'cortex', event: 'agent_complete', ...data}` @@ -672,7 +672,7 @@ def handle_action(): - If Callosum disconnected, message is dropped (with debug logging) - Returns `True` if queued, `False` if bridge not started or queue full -**Reference implementations:** `apps/import/routes.py`, `apps/observer/routes.py` +**Reference implementations:** `solstone/apps/import/routes.py`, `solstone/apps/observer/routes.py` --- @@ -701,11 +701,11 @@ def handle_action(): - `.workspace-content-wide` - Full viewport width, ideal for data tables and grids - Both include consistent padding and mobile responsiveness -**See:** `convey/static/app.css` for implementation details +**See:** `solstone/convey/static/app.css` for implementation details **Examples:** -- Standard: `apps/home/workspace.html`, `apps/todos/workspace.html`, `apps/entities/workspace.html` -- Wide: `apps/search/workspace.html`, `apps/activities/_day.html`, `apps/import/workspace.html` +- Standard: `solstone/apps/home/workspace.html`, `solstone/apps/todos/workspace.html`, `solstone/apps/entities/workspace.html` +- Wide: `solstone/apps/search/workspace.html`, `solstone/apps/activities/_day.html`, `solstone/apps/import/workspace.html` ### CSS Variables @@ -725,27 +725,27 @@ Use these in your app-specific styles to respond to facet theme. **Best practice:** Scope styles with unique class prefix to avoid conflicts. -**Example:** `apps/stats/workspace.html` shows scoped `.stats-*` classes for all custom styles in its `