# block-graph Specification

## Purpose

The core data model: blocks, outline hierarchy, pages, references, tags, and properties, with Loro-backed persistence and derived in-memory indexes.

## Requirements

### Requirement: Everything is a node
The system SHALL represent blocks, pages, tags, and journal dates as nodes in a single graph. Pages SHALL be nodes that own an ordered tree of block nodes; tags and dates SHALL be referenceable nodes with no special-cased storage.

#### Scenario: A tag is navigable like a page
- **WHEN** a block is tagged `#project`
- **THEN** `project` exists as a node whose backlinks include that block, exactly as a `[[project]]` page reference would

### Requirement: Stable block identity
Every block SHALL have a globally unique, stable identifier that survives edits, moves, reparenting, application restarts, and (future) sync merges.

#### Scenario: Identity survives restructuring
- **WHEN** a block is edited, indented under a new parent, and the application is restarted
- **THEN** the block retains its original identifier and all references to it resolve

### Requirement: Outline tree operations
The system SHALL support creating, deleting, splitting, and merging blocks, and moving blocks (with their subtrees) to any position under any parent, preserving sibling order.

#### Scenario: Subtree move
- **WHEN** a block with three descendant blocks is moved under a different parent
- **THEN** the block and all three descendants appear at the new location in their original relative order, and the source location no longer contains them

### Requirement: Loro document is the source of truth
The system SHALL persist the graph exclusively as a Loro CRDT document (movable tree for hierarchy, text for block content, maps for metadata), stored as a snapshot plus incremental update blobs in the graph directory. No derived structure may hold data that cannot be rebuilt from the Loro document.

#### Scenario: Derived indexes are disposable
- **WHEN** all derived index files are deleted and the application is restarted
- **THEN** the graph loads completely from the Loro document and all queries, backlinks, and search results are identical to their pre-deletion state

#### Scenario: Crash safety
- **WHEN** the process is killed immediately after an edit is acknowledged in the UI
- **THEN** on next startup the edit is present

### Requirement: Block references and backlinks
Block content SHALL support inline references to pages, tags, dates, and individual blocks. The system SHALL maintain a backlink index mapping every node to the set of blocks that reference it, updated within the same edit transaction.

#### Scenario: Backlink appears immediately
- **WHEN** the user types a reference to `[[fishing]]` inside a block and confirms it
- **THEN** the `fishing` node's backlink set includes the referencing block without an application restart or manual reindex

### Requirement: Lightweight properties
Blocks SHALL support key–value properties where values are typed as text, number, date, boolean, or node reference. Property keys SHALL be queryable across the graph. Property schemas, classes, and inheritance are explicitly out of scope for this change.

#### Scenario: Property round-trip
- **WHEN** a block is given the property `due` with the date value `2026-07-10`
- **THEN** a query for blocks with a `due` property returns the block, and the value is retrieved as a date (not a string)

### Requirement: Incremental index maintenance
The in-memory graph and all derived indexes SHALL be updated incrementally on every edit, and an incremental state SHALL be behaviorally indistinguishable from a full rebuild.

#### Scenario: Incremental equals rebuild
- **WHEN** a randomized sequence of at least 1,000 edit operations (create, edit, move, delete, tag, reference) is applied
- **THEN** the incrementally-maintained indexes are equal to indexes rebuilt from scratch from the Loro document

### Requirement: Graph directory format is versioned
The graph directory SHALL carry a semantic format version, written at graph
creation and readable without decoding the Loro document. Opening a graph
whose format version exceeds the running build's supported version MUST fail
with an error identifying both versions, without reading or modifying any
other file in the directory. A graph directory without a version stamp SHALL
be treated as format version 1 and stamped on next open.

#### Scenario: Newer format is refused untouched
- **WHEN** a build supporting format 2 opens a graph stamped format 3
- **THEN** the open fails with an error naming both versions and every file
  in the graph directory is byte-identical to its pre-open state

#### Scenario: Legacy graph acquires a stamp
- **WHEN** a graph directory created before versioning existed is opened
- **THEN** it is treated as format 1, opens normally, and carries a format
  stamp afterwards

### Requirement: Older formats migrate through a contiguous chain
Opening a graph with an older format version SHALL migrate it through
sequential compiled migration steps (each step raising the version by exactly
one) before normal loading, re-stamping after each step so an interrupted
migration resumes at the failed step. The registry of migration steps MUST be
verified contiguous from version 1 to the current version by a test. Before
the first migration step modifies the directory, the pre-migration snapshot
MUST be preserved as a backup.

#### Scenario: Migration chain has no gaps
- **WHEN** the current format version is N
- **THEN** a test asserts exactly N−1 registered migration steps, so a
  version bump without its migration fails the build

#### Scenario: Interrupted migration resumes safely
- **WHEN** the process dies between migration steps
- **THEN** the next open resumes migration from the recorded intermediate
  version and completes normally, and the pre-migration snapshot backup
  still exists

### Requirement: Cross-release open compatibility is continuously verified
The test suite SHALL include a committed golden graph directory that every
test run opens and verifies content-identical (block text, structure,
references) against a recorded expectation. The golden directory SHALL be
regenerated only together with a deliberate format version bump.

#### Scenario: Dependency upgrade cannot silently break decoding
- **WHEN** a change (including a Loro version bump) alters how existing graph
  files are decoded
- **THEN** the golden-graph test fails, forcing either a fix or an explicit
  format version bump with a migration

### Requirement: Update log is bounded by automatic compaction
The system SHALL automatically fold the incremental update log into a fresh
snapshot so that the log cannot grow without bound. Compaction SHALL trigger
(a) when, after an edit has been durably persisted, the update log has reached
a size threshold, and (b) at application quit when the update log exceeds a
smaller quit-time threshold. Compaction MUST NOT be required for correctness:
it only trades log-replay time at open for a snapshot write.

#### Scenario: Size threshold crossed during a session
- **WHEN** an acknowledged edit brings `updates.log` to or past the automatic
  compaction threshold
- **THEN** the log is folded into `snapshot.loro` and emptied (or removed), and
  a subsequent open of the graph directory yields the identical graph state
  while replaying no pre-compaction updates

#### Scenario: Quit-time compaction
- **WHEN** the application quits normally with a non-trivial `updates.log`
  (at or past the quit-time threshold)
- **THEN** on the next launch the graph loads from the snapshot with the
  pre-quit edits already folded in

#### Scenario: Trivial sessions do not rewrite the snapshot
- **WHEN** the application quits after a session whose `updates.log` is below
  the quit-time threshold
- **THEN** the snapshot file is not rewritten and the small log is simply
  replayed on next open

### Requirement: Compaction preserves the acknowledged-edit guarantee
Automatic compaction SHALL run only after the triggering edit is durable in the
update log, and a compaction failure or interruption at any point MUST NOT lose
an acknowledged edit, MUST NOT leave the graph directory unloadable, and MUST
NOT surface as a persistence failure for the edit itself. Before the update log
is removed, the newly written snapshot MUST be verified loadable, and the
previous snapshot MUST be retained (as `snapshot.loro.prev`) as a manual
recovery fallback. Opening a graph whose current snapshot fails to decode MUST
fail with an error identifying the retained fallback rather than silently
loading stale state.

#### Scenario: Kill between snapshot replacement and log removal
- **WHEN** the process is killed after compaction has atomically replaced
  `snapshot.loro` but before `updates.log` is removed
- **THEN** the next open loads the graph to the identical state (re-importing
  updates already contained in the snapshot is harmless), and a later
  compaction removes the stale log

#### Scenario: Compaction failure is not an edit failure
- **WHEN** the snapshot write fails (e.g. disk error) after the triggering
  edit was appended and fsync'd
- **THEN** the edit is still acknowledged as persisted, the previous snapshot
  and the update log remain intact and loadable, and compaction is retried at
  the next trigger

#### Scenario: Unverifiable snapshot never replaces a good one
- **WHEN** the newly exported snapshot fails verification (its bytes do not
  decode into a loadable document)
- **THEN** compaction aborts before touching the current snapshot or the
  update log, both remain intact and loadable, and the failure is reported
  without affecting the acknowledged edit

#### Scenario: Prior snapshot survives as a recovery fallback
- **WHEN** a compaction completes successfully
- **THEN** the pre-compaction snapshot remains on disk as
  `snapshot.loro.prev`, and a subsequent open that finds the current snapshot
  undecodable fails with an error naming that fallback instead of silently
  loading it

### Requirement: Fold state is document content
Each block SHALL support a boolean fold flag stored in the block's Loro
meta map (alongside `block_type` and `content`), persisted and merged with
the document like any other content, so that fold state survives restarts,
follows the document across (future) sync, and can be authored by (future)
templates. The flag SHALL NOT appear in the block's `properties` map,
property queries, or derived property indexes. An absent flag SHALL be
read as expanded; the flag's presence MUST NOT require a graph
format-version bump or migration.

#### Scenario: Fold flag round-trips through storage
- **WHEN** a block's fold flag is set to true and the graph is closed and reopened
- **THEN** reading the block's fold flag returns true, and blocks never flagged read as expanded

#### Scenario: Fold flag is invisible to the property system
- **WHEN** a block is folded and the graph's properties are queried
- **THEN** no property derived from the fold flag appears on the block, and property indexes are unchanged

#### Scenario: Pre-existing graphs are unaffected
- **WHEN** a graph created before the fold flag existed is opened by this build
- **THEN** it opens without migration at its recorded format version, and every block reads as expanded

### Requirement: Property-grid disclosure is document content
Each block SHALL support a boolean property-grid disclosure flag stored in
the block's Loro meta map beside the fold flag, persisted and merged with
the document like any other content. The flag SHALL NOT appear in the
block's `properties` map, property queries, or derived property indexes. An
absent flag SHALL be read as expanded; the flag's presence MUST NOT require
a graph format-version bump or migration.

#### Scenario: Disclosure round-trips through storage
- **WHEN** a block's grid disclosure is set to collapsed and the graph is
  closed and reopened
- **THEN** reading the flag returns collapsed, and blocks never flagged
  read as expanded

#### Scenario: Disclosure is invisible to the property system
- **WHEN** a block's grid is collapsed and the graph's properties are
  queried
- **THEN** no property derived from the disclosure flag appears on the
  block, and property indexes are unchanged