# Dogfood failures

This is a record of failures observed through the real local UI and daemon. It is not a claim that the underlying durability rules are wrong. The failure is that ordinary operator actions require knowledge of internal state that the product does not expose or repair.

## 2026-09-13: accepted input produced no reply

### What happened

1. A daemon started at 03:24 kept the model configuration it loaded at startup.
2. `klbr-daemon doctor` later validated the new on-disk DeepSeek, embedder, and reranker configuration, but did not identify that the live daemon still held the old `http://localhost:8001/v1` LLM route.
3. The UI durably admitted `hai`. The provider connection failed before any remote request. The original work item and its maintenance repair item both became terminal failures.
4. The UI showed neither failure as the answer to the input. The operator saw silence.
5. During diagnosis, `drain default` was used as if it meant service quiescence. It actually persisted a terminal `stopped` session. This operator mistake exposed that the CLI has `stop` and `drain`, but no corresponding session resume/recovery command.
6. The restarted UI accepted a click for the stopped session, cleared the composer, and rendered no error. The server returned an admission error over the WebSocket, but the UI did not preserve the attempted text or present a useful failure state.
7. The session was recovered offline without changing historical events or failed work items. A new input was admitted as an explicit retry.
8. The retry reached `https://api.deepseek.com/v1/chat/completions` and received HTTP 200. DeepSeek returned useful assistant prose but no Python tool call. The runtime correctly persisted ordinary assistant prose as scratchpad, so no `local.send` speech was published. The operator still received no reply.

### Evidence retained

- Events 1-9 retain the first input, both `localhost:8001` connection failures, and their context receipts.
- `work-input-1` and `work-act-d779496a1307a22e` remain failed.
- Event 10 is the explicit retry. Events 12-15 retain its context receipt, model output, request usage, and completed work item.
- No speech event exists for either input.

## Why this was too complicated

The runtime has precise internal distinctions but no small operator-facing state machine. A person sending a message should not need to understand daemon config snapshots, event cursors, work-item terminality, session registry residency, or the difference between service quiescence and operator drain.

Keep the durable internals, but project them into one visible input lifecycle:

- `queued`
- `running`
- `replied`
- `held(reason)`
- `failed(reason, retryable)`

Every accepted input must reach one of those visible outcomes. A failure must stay attached to its input. Retry must create a linked new attempt, never mutate or silently duplicate the old one.

## Required fixes before calling the UI dogfoodable

1. **Live config identity:** `status` and the UI show daemon start time plus the loaded config fingerprint/routes. `doctor` reports when disk config differs from the live process.
2. **One restart operation:** service restart performs quiescence without changing a session to operator-stopped. `drain` must say that it persistently stops the session before confirmation.
3. **Explicit session recovery:** add an operator `resume`/`start` command and UI action for a stopped session. It must start a fresh runtime owner while retaining history.
4. **Input-correlated failures:** render admission, provider, policy, and execution failures on the input card. Do not clear unrecoverable input text without an acknowledged durable admission.
5. **Explicit retry:** offer retry only when the recorded failure proves it is safe or requires operator confirmation. Link the retry to the failed attempt.
6. **No-speech outcome:** an operator input whose turn completes without confirmed speech must visibly say so. For an ordinary interactive reply, the model contract must make `await local.send(...)` unavoidable or the runtime must return a typed `no_speech` outcome; scratchpad prose must not look like success.
7. **Reconnect truth:** replay the completed durable lifecycle after reconnect. Lossy token animation is fine; terminal disposition is not.

## Implemented repair

The default UI is now an actual conversation: operator inputs and confirmed `local.sent` speech. Each input carries an exact durable `turn.outcome` or `turn.failed` association by `input_event_ids`; historical records without one use a bounded compatibility fallback. Scratchpad, provider reasoning, receipts, usage, kernel lifecycle, slices, tool results, and raw JSON remain available behind explicit scratchpad and diagnostics views.

The web client requests history from sequence zero when its local timeline is empty. At turn completion the daemon broadcasts the complete committed tail, so a connected client cannot advance its cursor past durable records it never received. Durable model records replace lossy live fragments rather than merging unrelated model steps.

A shared model-visible runtime contract restores the old explicit-action rule without restoring implicit delivery: plain assistant content is scratchpad, operator speech requires `await local.send(...)`, and deliberate waiting requires `await runtime.wait(...)`. A typed `no_reply` outcome remains visible when a model still completes without confirmed speech.

## Prompt regression confirmed

The last known-good pre-v2 prompt is `klbr-core/src/instructions.md` at commit `f64ebc3`. The old `Agent` appended it to the soul and explicitly taught that plain assistant text is secret scratchpad, named each send/wait action, and included good/bad examples. The old turn loop also detected prose without `local_send`, injected a corrective nudge containing that prose, and continued inference.

The exact system content used by retry event 10 has SHA-256 `c612d369ab03c03624d83662ffb26245e7617b0d370cc4f752090906358af28d`, matching only `DEFAULT_SOUL + standing-responsibility.md`. It omits the old scratchpad/event-loop contract. The Python tool description mentions `await local.send(...)`, but a no-tool response now immediately completes the turn. This is why the model believed no tool call was required.

This regression is now repaired at the producer boundary. `context_receipt` and `request_usage` payloads are stored as `context.receipt` and `model.usage`; tool results and other lifecycle records also have typed event kinds under migration 014. Historical scratchpad-encoded records remain immutable and are projected compatibly as diagnostics.

## Verification

- Rust: every test in the 134-test workspace run is green after updating the one obsolete assertion that required `turn_failed` to remain a scratchpad record; the final daemon v2 integration run passed 10/10.
- Python runtime: 114/114 passed.
- Web production build: passed after the final client changes.
- Real DeepSeek/UI smoke: input event 42 produced confirmed `local.sent` event 48 and `turn.outcome` event 54 with status `replied` and `input_event_ids: [42]`. The same live client rendered the reply in conversation view and the complete durable diagnostic tail separately.

## Simplification rule

Do not weaken exactly-once admission or turn scratchpad into implicit speech. Remove the need for operators and clients to reconstruct those mechanisms. One input, one visible lifecycle, one explicit recovery path.