diff --git a/openspec/changes/add-supertags/.openspec.yaml b/openspec/changes/add-supertags/.openspec.yaml new file mode 100644 --- /dev/null +++ b/openspec/changes/add-supertags/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-07-15 diff --git a/openspec/changes/add-supertags/design.md b/openspec/changes/add-supertags/design.md new file mode 100644 --- /dev/null +++ b/openspec/changes/add-supertags/design.md @@ -0,0 +1,170 @@ +# Design: add-supertags (exploration record) + +> Direction captured from the remoras review (2026-07-14/15) before +> implementation planning. Decisions below are settled *in direction*; open +> questions at the bottom are the known unknowns to resolve when this becomes +> implementable. Remoras references are to the corpus at +> https://tangled.org/byarielm.fyi/remoras (`llm/design/03-data-model.md`, +> ADR-010/ADR-020). + +## Context + +Trawler has lightweight typed properties (text/number/date/bool/node-ref) with +schemas explicitly out of scope, tags as first-class graph nodes, and a +capability-scoped Steel VM powering query blocks. Remoras' data model doc is +the most concrete picture of the schema layer trawler deferred: supertags +(tags with schemas), an ad-hoc→typed upgrade path, and a computed-property +layer. Its execution has flaws worth designing around (see D1, D5). + +## Goals / Non-Goals + +**Goals:** + +- Record the supertag direction: emergent schemas on tag nodes, lexicon- + compatible representation, Scheme authoring surface. +- Record the query-evaluation direction that unlocks computed properties: + stale-while-revalidate + dynamic read-set tracing. +- Record the deliberate divergences from remoras and why. + +**Non-Goals:** + +- Implementable specs/tasks (deferred; see proposal status note). +- Rollups/formulas/views themselves — this records their *enabler*, not their + design. +- Any atproto networking. Lexicon compatibility is a shape constraint on the + schema value, nothing more, until a sharing feature exists. + +## Decisions + +### D1 — Schemas attach to tag nodes; the tags-are-nodes model stays + +A supertag is a tag node that carries a schema. No new "supertag" entity, no +tags-vs-pages split. Remoras separates tags (untyped labels) from pages, then +needs a same-named page again the moment a tag gets a schema — the split buys +a UI fix (empty-page pollution) at the cost of a forked model. Trawler's +uniform node model gets supertags for free: the tag already exists as a +referenceable node with backlinks; the schema is data attached to it. +Empty-page pollution stays a rendering concern (uncreated views already render +distinctly). + +### D2 — Schema representation is lexicon-compatible from day one + +The canonical schema value is declarative JSON-shaped data whose field types +map onto ATProto lexicon primitives: text→string, number→integer/float, +date→string(datetime), boolean, select→string+knownValues, multi-select→array. +Two property kinds are deliberately *excluded* from the shareable schema: + +- **Relations** don't travel across graphs — a schema may declare "references + a `#Project`-tagged node" but targets are local block IDs. +- **Formulas/rollups** are behavior, not data schema. They stay local (and + arrive with the computed-property layer, not this change). + +Sharing model, when it comes: schemas-as-records first (one trawler-owned +lexicon whose records describe schemas; publish/install = PDS record +put/fetch), with schemas-as-published-lexicons (own NSIDs, cross-app interop) +as a possible later upgrade. Sharing is atproto-public by definition; graph +content never goes there (same hard boundary as remoras ADR-009). + +### D3 — Schemas are authored in-editor as quotable Scheme + +A schema is a block whose content is a Steel s-expression — same pattern, +highlighter, and capability-scoped VM as query blocks: + +```scheme +(schema + (field "status" (select "todo" "doing" "done")) + (field "due" 'date) + (field "project" (relation 'Project))) +``` + +Hard constraint: **data semantics only**. The schema DSL must be quotable — no +conditionals, no computation producing fields — so evaluation is total, +deterministic, and the value round-trips losslessly to the lexicon-compatible +JSON form (D2). The moment schemas can be computed, export becomes partial and +schema identity unstable. Evaluation produces a validated schema value; that +value (not the source text) is what typing/validation and export consume. +This fixes remoras' weakest execution detail (schema as a UI-only prop blob, +un-editable in the outliner). + +### D4 — Ad-hoc properties upgrade to schema-backed, never break + +Remoras' best idea, adopted as-is: `key:: value` properties work with no +schema (stored, searchable, best-effort typed at query time). When a supertag +schema later defines the key's type, existing values are validated +retroactively with warnings, not errors. Capture never has prerequisites; +structure is always retrofittable. + +### D5 — Multiple supertags compose by union; conflicts are surfaced, not +silently ordered + +A block may carry several supertags; its effective schema is the union. +Remoras resolves key conflicts by "first supertag wins" — a silent footgun +where tag order changes semantics. Direction here: identical-key-identical-type +merges silently; identical-key-different-type is surfaced as a schema conflict +on the block (visible state, like a validation warning). Inheritance +(`extends`-style) is deferred entirely — union composition covers the known +use cases and single inheritance can be added compatibly later. + +### D6 — Query evaluation: stale-while-revalidate + dynamic read-set tracing + +The enabler for computed properties, applicable to query blocks immediately: + +- **SWR**: cached results render instantly; revalidation runs off the + interaction path; edits never wait on query cost. +- **Traced invalidation**: queries can touch the graph only through the + registered primitives (`tag`, `ref`, `prop`, `search`, `descendants`, + block reads inside `filter`/`map`) — so instrument them to record each + evaluation's read-set. An edit invalidates a cached result only if it + touches that read-set (search-based queries conservatively invalidated by + any text change, refinable later). Sound by construction: the sandbox + already enforces determinism and mediated access. This is the dynamic- + dependency (Salsa/build-trace) alternative to remoras' restricted formula + DSL — the restriction exists to make dependencies statically derivable; + tracing makes it unnecessary, so Scheme stays the one language. +- **Cache ID sets, render content live**: staleness is then membership-only + (a row lingers/missing briefly), never stale text. A subtle "revalidating" + affordance on the query block covers the gap. +- Known limit: tracing decides *when* to recompute, not how to recompute + less — an invalidated query re-runs fully. Acceptable at personal-graph + scale; same effective granularity as remoras' reactive engine. +- **Correctness gate** (adopted from remoras spec 19's rebuild-equivalence + pattern, applied to the query cache): a property test asserting + *traced incremental revalidation ≡ full re-evaluation* — over random op + sequences against fixture graphs, the SWR cache after traced invalidations + must equal evaluating every query from scratch. This is the exit test the + SWR change must pass before shipping; it is to the query cache what + index-rebuild equivalence is to a derived index. + +## Risks / Trade-offs + +- [Schema DSL scope creep toward computation (defaults from expressions, + conditional fields)] → the quotability constraint (D3) is the line; anything + computed belongs to the future formula layer, not the schema. +- [Read-set tracing of `search` is coarse — any text edit invalidates] → + accepted initially; refine with index-level changed-doc info if it thrashes. +- [Background revalidation needs Steel evaluated off the UI thread against a + stable index view] → the main unresolved implementation question (snapshot + the index vs. idle-slice evaluation on the main thread); resolve before the + query-eval change is specced. +- [Lexicon compatibility constrains property types forever] → deliberate; the + excluded kinds (relations, computed) are excluded for reasons that hold + regardless of atproto. + +## Open Questions + +1. Where does the schema value live — a property on the tag node, or derived + from the schema block on demand (source of truth: Scheme text vs. evaluated + value)? Interacts with SWR if schemas are re-evaluated. +2. Schema versioning/migration when a shared (or local) schema changes under + existing typed values. Default policy adopted from remoras spec 16 unless + implementation finds otherwise: per-schema version (not global); *adding* a + field needs no migration (missing = unset); *removing* a field orphans + existing values harmlessly (never delete user data); *renaming* is the only + operation that touches blocks. Remaining open part: what an *imported* + (shared-record) schema update does — auto-apply, prompt, or pin-to-version. +3. Validation UX: where do type violations surface (block, tag page, panel)? +4. Does `select` need option metadata (color, order) and does that survive the + lexicon mapping (knownValues carries no metadata)? +5. Threading model for background revalidation (see Risks). +6. Split point: does D6 land as its own change before supertags (likely), and + do query blocks adopt SWR before any schema work begins? diff --git a/openspec/changes/add-supertags/proposal.md b/openspec/changes/add-supertags/proposal.md new file mode 100644 --- /dev/null +++ b/openspec/changes/add-supertags/proposal.md @@ -0,0 +1,66 @@ +# Proposal: add-supertags + +> **Status: exploration record.** This change captures a settled *direction* +> from the July 2026 remoras design-corpus review, ahead of implementation. +> Specs and tasks are deliberately not written yet; when implementation nears, +> the query-evaluation half (SWR + read-set tracing) likely splits into its own +> change that lands first. Sources: remoras `design/03-data-model.md` and +> `design/01-decisions-adr.md` (ADR-010, ADR-020), examined 2026-07-14/15; +> conversation decisions recorded in this file and `design.md`. + +## Why + +Trawler's block-graph spec deliberately scoped out "property schemas, classes, +and inheritance." Supertags — tags whose blocks gain typed, schema-backed +properties — are the wanted shape of that future layer: they make +capture-now-structure-later native (free-form blocks upgrade to typed ones by +tagging), and they're the foundation for table/board views and computed +properties. The remoras review produced a concrete design direction worth +recording before the details fade, including two deliberate divergences from +remoras and one enabling idea for query evaluation. + +## What Changes + +- **Supertags**: a tag node can carry a schema; blocks tagged with it gain the + schema's typed properties. Emergent, not ceremonial — attaching a schema to + the tag node is what makes it a supertag. Trawler's tags-are-nodes model is + kept (schema hangs off the existing tag node; no tags-vs-pages split, unlike + remoras). +- **Lexicon-compatible schemas**: the schema representation is declarative, + JSON-serializable, and maps onto ATProto lexicon primitives from day one, so + a schema can later be published/installed as an ATProto record (sharing + layer only — graph data never touches atproto). +- **Scheme authoring surface**: schemas are written in-editor as Steel + s-expressions with data semantics only (quotable, no computation), evaluated + in the existing capability-scoped VM to a validated schema value — the same + block-whose-content-is-Scheme pattern as query blocks. +- **Query evaluation model** (enabler, likely its own change): cached query + results served stale-while-revalidate, invalidated precisely via dynamic + read-set tracing of the existing query primitives. This is what later makes + Scheme-based computed properties/rollups viable without a restricted + formula DSL. + +## Capabilities + +### New Capabilities + +- `supertags`: schema definition on tag nodes, property typing/validation, + ad-hoc→schema-backed upgrade path, schema authoring blocks, lexicon + export/import shape. (Spec deferred until implementation.) + +### Modified Capabilities + +- `block-graph`: lifts the "schemas explicitly out of scope" carve-out on the + lightweight-properties requirement. (Delta deferred.) +- `steel-queries`: evaluation gains SWR caching + traced invalidation; the + schema DSL joins the query DSL as a second capability-scoped Steel surface. + (Delta deferred; likely splits into its own change.) + +## Impact + +- `trawler-core`: schema model + validation, read-set tracing in the query + primitive registrations, query result cache. +- `trawler`: schema-block rendering/editing (reuses scheme highlighting), + typed property affordances, staleness indicator on query blocks. +- No dependency on any sync/atproto code — lexicon compatibility is a data- + shape constraint now, network features later.