feat(observe): bounded JSON extractors for messaging + calendar frames master
Messaging and calendar screen categories now emit strict, bounded JSON with category formatter modules that render dict outputs back into the familiar transcript shape. Category discovery also carries a per-category output-token cap; categories without a frontmatter override still use 4096. D3 token derivation: - Real-data basis (n=1304 content.messaging frames, real journal): total chars median 1055 / p90 2522 / p99 4389 / max 7303; messages per frame max 41; longest single message text 2859 chars; sender max 64; timestamp max 65. - Bounds sit ABOVE observed maxima on purpose: messages maxItems 60 (>41), text maxLength 4000 (>2859), sender 128 (>64), timestamp 96 (>65). schema_prep.py strips maxItems/maxLength from Anthropic requests, but models.py still validates the response against the canonical bounded schema, so a bound sized at the median is a production SchemaValidationError, not a lint failure. - Output cap: a faithful worst-case extraction re-encoded as JSON is ~7303 chars of content plus key/quote/escape overhead (~1.45x), about 10.6k chars, or about 3.0k tokens at 3.5 chars/token. The previous global cap of 4096 output tokens (~14.3k chars) left about 26% headroom over the observed p100, and JSON truncation is fatal (JSONDecodeError -> retry -> error) where markdown truncation degraded gracefully. Messaging and calendar therefore get max_output_tokens: 8192 via new category frontmatter while the default stays 4096: about 2.7x the observed p100 and about 6x p99. - Explicitly rejected: proving the schema's theoretical maximum serialization fits 4096 tokens. For an array-of-free-text schema that ceiling is maxItems x maxLength (60 x 4000 = 240k chars) and is not a useful bound; forcing it under 14.3k chars would cap text near 200 chars and truncate real 2859-char messages. Hence the per-category frontmatter key rather than a tighter schema. - Side effect, intended: both categories now take the JSON thinking_budget of 6144 (was 4096) via the existing 6144 if is_json else 4096 path in describe.py. D4 no migration: - 1304 on-disk frames keep content.messaging as a markdown str. Both formatters return "" for non-dict input, routing legacy frames through the existing str fallback in screen.py::_format_category_content (**Messaging:** plus the raw body), byte-identical to main. This is the meeting.py contract, not a compat shim. No migration script, no data rewrite. Coverage honesty: - calendar has ZERO real frames in the journal; calendar.md was added in 97a97a7b and has never produced an extraction. Its coverage is synthetic fixtures only. - make eval-schemas was NOT run: no local provider is configured, so tests/eval_schemas.py raises LOCAL_NOT_READY. No qwen results are claimed. scripts/check_schema_bounds.py::ALLOWLIST is unchanged. The four talent schemas labelled messaging/calendar follow-on lode (chat, participation, participation_entry, schedule) are talent schemas, out of scope for this lode, and remain allowlisted. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>