Trawler #
A personal, keyboard-first, Rust-native outliner in the mold of Logseq DB: a CRDT-backed block graph, a journal-first navigation model, and a Steel (Scheme) query engine for live, first-class queries over your notes.
Built with GPUI (Zed's UI framework), Loro (CRDT storage), Steel (embedded Scheme), and tantivy (full-text search).
Status #
This is the trawler-mvp change (see openspec/changes/trawler-mvp/) — a
single-user MVP, not a released product. Phases 1–7 (block graph core, search,
Steel queries, the outline editor, journal/navigation, and query blocks) are
implemented; Phase 8 (hardening) is in progress. See
openspec/changes/trawler-mvp/tasks.md for the exact task-by-task status and
design.md for the architecture decisions and known deviations from the
original spec.
Building and running #
Requires the Rust toolchain pinned in rust-toolchain.toml (currently
1.94.0, x86_64-pc-windows-msvc) — rustup will pick it up automatically.
Windows is the only platform this has been built and run on; GPUI's Windows
backend is confirmed to use real Direct3D 11 rendering (see design.md Spike
S1), but other platforms are untested.
cargo run -p trawler
On first launch, this creates a graph directory and opens straight to today's journal page, focused and ready to type.
The UI font is Inter and the monospace font (code
spans, code blocks, Scheme query source) is
Paper Mono — both SIL
OFL 1.1, bundled into the binary from crates/trawler/assets/fonts/ and
registered at startup. No system font installation needed, and every
platform renders identically.
Development commands:
cargo check --workspace
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace
cargo fmt --all
NixOS / Linux development shell #
The repository flake provides the pinned Rust 1.94 toolchain and the native X11/XCB, xkbcommon, font, Wayland, and Vulkan dependencies needed to build the full GPUI workspace on x86_64 Linux. It does not install Trawler or change the host system:
nix develop
cargo check --workspace --all-targets
cargo test --workspace
The devshell sets RUST_TEST_THREADS=1 for the default test command. The
Steel interruption gate measures a strict wall-clock budget and can become
spuriously slow when it runs alongside other CPU-heavy debug tests. Override
the variable for a faster best-effort parallel run.
The shell exports TRAWLER_LAVAPIPE_ICD, the store-pinned Mesa software
Vulkan driver used by the isolated preview helper. It deliberately does not
override VK_DRIVER_FILES merely by entering the shell, so a manual launch
on a real Linux desktop can continue to use the desktop's hardware renderer.
The Linux environment is for development and automation. Windows 11 remains the product target and the authoritative platform for final visual/interaction checks.
Some tests are expensive (they build synthetic 100k-block graphs) and are
#[ignore]d by default:
cargo test -p trawler-core --release -- --ignored
Data-integrity gates #
Two seeded gates (openspec capability integrity-gates) run in the default
cargo test --workspace and make the data-safety promises executable:
- Rebuild equivalence (
tests/rebuild_equivalence.rs): after a random op stream withGraphIndexand the tantivy index maintained incrementally (exactly as the app maintains them), rebuilding both from the Loro doc alone must answer identically — backlinks, tags, properties, structure, and search hits. The comparator (integrity::ObservableState) is the working definition of "derived state"; extend it when adding a new derived structure. - Crash consistency (
tests/crash_safety.rs): a child process applies and persists seeded edits, is hard-killed (TerminateProcess— no cleanup) at a random moment, and on reopen every acknowledged edit must be present with derived state matching a deterministic reference replay.
Both print their seed; reproduce any failure exactly with
TRAWLER_TEST_SEED=<seed>. Deeper sweeps are #[ignore]d alongside the
other expensive tests.
A third always-on gate protects format compatibility: the golden graph
(crates/trawler-core/tests/golden/graph, exercised by
tests/golden_graph.rs) is a small committed graph directory every test
run must open and read identically. It is regenerated only via the
#[ignore]d regenerate_golden_graph test, and only together with a
deliberate format version bump.
Loro upgrade policy #
The loro dependency stays pinned to an exact version. An upgrade PR must
show the golden-graph test passing unmodified, plus a round-trip check
(open the golden graph → export a snapshot with the new Loro → reopen →
identical content). If a Loro upgrade cannot read existing snapshots, that
is by definition a graph format change: bump CURRENT_FORMAT_VERSION,
register the migration (e.g. re-export via the previously pinned version),
and regenerate the golden graph in the same commit — the
migration_chain_is_contiguous test enforces the pairing.
Where your data lives #
Without an override, Trawler uses the platform-native data directory:
- Windows:
%APPDATA%\trawler\graph(roaming AppData) - macOS:
~/Library/Application Support/trawler/graph - Linux:
$XDG_DATA_HOME/trawler/graph, or~/.local/share/trawler/graphwhenXDG_DATA_HOMEis unset
A present, non-empty TRAWLER_GRAPH_DIR has highest precedence and bypasses
both the native default and legacy compatibility checks. This is useful for
selecting another graph or a development scratch graph:
TRAWLER_GRAPH_DIR=/tmp/my-test-graph cargo run -p trawler
An ordinary relative value is intentional: Trawler resolves it against the
launch working directory (without following symlinks) and uses the resulting
absolute path. On Windows, drive-relative values such as C:graph and rooted
values without a drive or UNC prefix such as \graph are refused because they
do not identify a stable absolute location; use C:\graph, a UNC path, or an
ordinary relative value instead. Use an absolute override if the launch working
directory might not be available. A present but empty TRAWLER_GRAPH_DIR is
not treated as unset; this is a deliberate compatibility break, and startup
refuses it before opening storage. Remove the variable to use normal selection,
or set it to a non-empty path.
Older builds could create trawler-graph under the launch working directory.
With no override, Trawler opens such a legacy graph only when it contains a
regular snapshot.loro and the native graph does not; startup warns that
legacy compatibility was used and names both locations. Trawler never creates,
moves, or copies a legacy graph automatically. If both the native and legacy
locations contain different recognized graphs, startup refuses the ambiguous
choice. Recover by setting TRAWLER_GRAPH_DIR to the absolute path of the one
you intend to open; do not delete or merge either graph merely to get past the
check.
Trawler also refuses to initialize an unrecognized native directory that has
unrelated files or any updates.log content. Do not empty it in response to the
error: close Trawler, preserve and back up the complete directory, inspect its
origin, and either restore/move the intended complete graph or select a separate
graph with an absolute override. The only automatic retry exception is the
narrow debris from an interrupted first creation: a directory containing only
meta.json and/or snapshot.loro.tmp can be initialized again. Any other file
keeps the directory occupied.
To migrate a legacy graph conservatively:
- Close every Trawler process that could write either location.
- Back up the complete legacy graph directory and confirm the backup is readable before changing the original.
- Move the complete directory to the native path, or copy it if you want to preserve the original during verification. Do not combine it with an existing native graph.
- Launch with
TRAWLER_GRAPH_DIRexplicitly set to the absolute native destination, verify the expected pages and recent edits, then close Trawler. - After successful verification, rename/archive the complete legacy directory
so default startup no longer sees two graphs. Never retire only
snapshot.loro: it is source-of-truth data and leaving the rest of the directory behind creates an incomplete, misleading copy. If you intentionally keep both complete recognized graphs, keep using an explicitTRAWLER_GRAPH_DIRon every launch.
The headless Linux preview sessions always use helper-owned, disposable fixture graphs via an explicit override; those isolated Nix scratch graphs are separate from this user-default and legacy selection policy.
Dev automation (devtools) #
For development — especially agent-assisted development — trawler has an optional automation mode: headless UI tests, a deterministic fixture graph, and a local socket for driving and observing a running instance. None of it exists in a default build.
UI tests #
cargo test --workspace includes gpui integration tests
(crates/trawler/src/ui_tests.rs) that open the real TrawlerApp over a
seeded fixture graph on gpui's fake test platform — no window, no GPU — and
drive it through the real keymap with simulated keystrokes. They run
identically on every OS and in CI.
Fixture graphs #
trawler-core's fixtures feature provides a deterministic seeded graph
(fixed dates, fixed content, even stable block ids) shared by the UI tests
and interactive dev sessions, so state dumps and screenshots are comparable
across machines. Seed one from the CLI (devtools build):
cargo run -p trawler --features devtools -- --seed-fixtures /tmp/scratch-graph
Automation server #
Build with --features devtools and launch with TRAWLER_DEVTOOLS=1 (both
required — a devtools binary run normally opens no sockets):
TRAWLER_DEVTOOLS=1 TRAWLER_GRAPH_DIR=/tmp/scratch-graph \
cargo run -p trawler --features devtools
The app binds a TCP listener on 127.0.0.1 (OS-assigned port, printed to
stdout and written to <graph-dir>/devtools.port) speaking one JSON object
per line, one response per line:
| Request | Effect |
|---|---|
{"cmd":"keys","keys":"ctrl-k"} |
Dispatch keystrokes through the real keymap (Keystroke::parse syntax, whitespace-separated). No OS focus needed. |
{"cmd":"type","text":"hello [["} |
Insert text through the focused editor's input path (triggers completion, query re-eval, etc.). |
{"cmd":"dump"} |
Versioned JSON of semantic UI state: view, visible rows, focused block/cursor/selection, open popups, window bounds. |
{"cmd":"bounds"} |
Window position/size/scale only. |
{"cmd":"screenshot","path":"shot.png"} |
Capture the app window to a PNG. |
Failures answer {"ok":false,"error":...} and never affect the app. Replies
are sent after the dispatched action has run; poll dump for state that
settles asynchronously (e.g. query re-evaluation after its debounce).
Screenshots are best-effort per platform: solid on Windows and X11, macOS needs a one-time Screen Recording permission, and Linux Wayland depends on the compositor — everything else keeps working where capture doesn't. (Implementation note: capture runs in a short-lived helper process because xcap won't enumerate the calling process's own windows on Windows.)
Headless Linux preview sessions #
From nix develop, tools/trawler-preview.py runs the real GPUI application
on a private Xvfb display with Mesa lavapipe and a minimal window manager
(required for xcap's window discovery). It owns a fresh fixture graph; it
never reads or infers the default graph directory.
Run the deterministic end-to-end smoke path (multi-step navigation, semantic dump, PNG screenshot, cleanup):
tools/trawler-preview.py smoke
The command prints the preserved artifact directory under
target/dev-preview/, containing initial-dump.json, final-dump.json,
screenshot.png, and build/application/window-manager/Xvfb logs.
For iterative work, keep one application open while sending requests and inspecting successive states:
tools/trawler-preview.py start
tools/trawler-preview.py status
tools/trawler-preview.py send --json '{"cmd":"keys","keys":"ctrl-k"}'
tools/trawler-preview.py send --json '{"cmd":"type","text":"trawler-design"}'
tools/trawler-preview.py send --json '{"cmd":"keys","keys":"enter"}'
tools/trawler-preview.py send --json '{"cmd":"dump"}' > /tmp/trawler-dump.json
tools/trawler-preview.py screenshot /tmp/trawler.png
tools/trawler-preview.py stop
stop is idempotent. status reports stale session metadata after a hard
child-process exit, and stop then removes only helper-owned processes and
scratch state. It never uses a global pkill, so a separately installed
desktop Trawler process is unaffected.
The devtools protocol has no raw mouse/pointer command on the current GPUI pin. Use keyboard equivalents for agent-driven flows; verify pointer-only affordances manually on a native desktop.
To run a visible development build from an interactive Linux desktop, still use a scratch graph rather than your real one:
scratch="$(mktemp -d)/fixture-graph"
cargo run -p trawler --features devtools -- --seed-fixtures "$scratch"
TRAWLER_GRAPH_DIR="$scratch" TRAWLER_DEVTOOLS=1 \
cargo run -p trawler --features devtools
This native path uses the current desktop and renderer. The automated helper always uses its own display, software renderer, binary, graph, and process lifecycle, so both can coexist without sharing a graph lock.
If preview startup fails, inspect the printed artifact directory. build.log,
seed.log, app.log, wm.log, and xvfb.log identify the failing stage. The
helper bounds waits for Xvfb, Trawler, and devtools.port instead of hanging
indefinitely.
Graph directory format #
Everything under the graph directory is derived from, or is, a single Loro
CRDT document — there is no separate database. Deleting any file except
snapshot.loro/updates.log and re-launching rebuilds it from scratch.
<graph-dir>/
snapshot.loro # the last compacted full snapshot of the Loro doc
updates.log # length-prefixed incremental update blobs since the
# last snapshot — replayed on top of it when opening
meta.json # semantic format version stamp (format-gating fields
# only), checked before anything else is read
search-index/ # tantivy full-text index; entirely disposable,
# rebuilt automatically if missing
snapshot.loro+updates.logare the only source of truth. Every other file (the search index) can be deleted safely; it's rebuilt transparently on next open.- The graph format is versioned (
meta.json, currently format 1). A graph stamped with a newer format than the running build is refused with a clear error, touching nothing — upgrade trawler to open it. Older formats migrate automatically through a sequential, test-gated migration chain (with the pre-migration snapshot kept assnapshot.loro.v<N>.bak); a graph from before versioning existed is treated as format 1 and stamped on next open. - The Loro document holds one movable tree (container name
"outline"). Tree roots are pages; everything else is a block. A block's text lives in its metadata map under thecontentkey (a mergeable Loro text); typed properties (used by query-blocktable/prop/prop-eq, and thequeryflag that marks a block as a query block) live underproperties. - Writes are append-only and crash-safe:
updates.logentries are length-prefixed,fsync'd before a write is considered acknowledged, and a torn trailing write (a kill mid-append) is detected and simply dropped on next open rather than treated as corruption. - The update log compacts automatically (openspec change
add-auto-compaction). Two triggers: when a persisted edit brings
updates.logto 4 MiB, it is folded into a fresh snapshot immediately; and on graceful app quit, any log over 64 KiB is folded so the next launch loads almost purely from snapshot. Compaction never replaces the only good copy: the new snapshot is verified loadable before the swap, and the pre-compaction snapshot is retained assnapshot.loro.prev— a manual recovery fallback (ifsnapshot.loroever fails to load, the error names it; note it predates the edits folded into the newer snapshot, so recovering from it loses that window). The log is also imported as a single batch on open, so even an uncompacted log replays quickly.
Keyboard reference #
Global:
| Key | Action |
|---|---|
Ctrl+K |
Quick open — fuzzy switcher over pages, tags, and journal-date shortcuts |
Ctrl+F |
Full-text search |
Alt+Left / Alt+Right |
Navigation back / forward |
Ctrl+. / Ctrl+, |
Zoom into / out of the focused block |
Ctrl+Enter |
Create a page from an uncreated tag/page/date view |
Ctrl+Shift+Q |
Toggle the focused block as a query block |
Ctrl+Shift+B |
Toggle the sidebar (Calendar and Similar panels) |
Ctrl+Shift+C |
Reveal the calendar in the sidebar (reset to the current month); toggles the sidebar closed if it's already open |
Ctrl+Shift+F |
Fold/unfold the focused block's children |
Every block renders a single Logseq-style bullet: clicking the bullet of a
block with children folds/unfolds them (same as Ctrl+Shift+F), and a
folded block's bullet gains an outline ring so hidden children stay
discoverable.
Back/forward, the calendar, and the sidebar toggle are also available as icon buttons in the titlebar for mouse-driven navigation. The calendar lives in the sidebar: click a day to jump to (or create) its journal page, click the month arrows to change months. Days that already have a journal page are bolded; today is highlighted. The sidebar is resizable by dragging its edge, and its panels keep their state while collapsed.
While editing a block:
| Key | Action |
|---|---|
Enter |
Split into a new sibling block at the cursor |
Shift+Enter |
Insert a newline within the block |
Tab / Shift+Tab |
Indent / outdent under the previous sibling |
Backspace at start of block |
Merge into the end of the visually previous block (the row rendered directly above) |
Alt+Up / Alt+Down |
Move the block up/down among its siblings |
Up / Down at a content boundary |
Move focus to the previous/next visible block |
Ctrl+Enter |
Follow the reference at/nearest the cursor |
[[ / # |
Open reference/tag completion |
Escape |
Dismiss an open completion popup |
Standard text editing (arrows, shift-select, Ctrl+A/C/V/X, Home/End)
works as expected within a block.
Query blocks #
Any block can become a query block: press Ctrl+Shift+Q while it's
focused, then write a Steel (Scheme) expression as its content. The result
renders live beneath it, re-evaluating a few hundred milliseconds after you
stop typing, without leaving the block.
A query evaluates in a read-only, capability-scoped Steel VM — only the primitives below are registered; there is no way to mutate the graph, touch the filesystem, or reach the network from a query. A runaway (non-terminating) query is interrupted after its time budget and shown as "Query timed out" rather than freezing the app.
Primitives #
Each of these returns a set (as a sorted list of block ids) unless noted:
| Primitive | Returns |
|---|---|
(tag 'name) |
Blocks tagged #name |
(ref "name-or-id") |
Blocks referencing that page, tag, date, or block id |
(prop "key") |
Blocks with property key set |
(prop-eq "key" "value") |
Blocks where property key equals value |
(date-range "2026-07-01" "2026-07-31") |
Blocks referencing a date in that range |
(descendants "peer@counter") |
Descendants of the given block id |
(search "text") |
Full-text search hits for text |
(and set set) / (or set set) / (not set) |
Native set intersection/union/complement |
(table set '("col1" "col2")) |
Projects each block's named properties into a table result instead of a plain list |
(filter (lambda (id) ...) set) / (map f set) |
Per-block Scheme predicate/transform over an already-narrowed set |
Plus a small stdlib: car, cdr, cons, null?, bool-not (note: not is
the set-complement primitive above, not boolean negation — use bool-not for
that), string-append, string-contains?, string-equal?, and
+ - * / = < > <= >=.
Examples #
; Everything tagged #project that also references [[rust]]
(and (tag 'project) (ref "rust"))
; Same set, projected as a table with due/priority columns
(table (tag 'project) '("due" "priority"))
; Blocks referencing a date in July, excluding anything tagged #done
(and (date-range "2026-07-01" "2026-07-31") (not (tag 'done)))
; Full-text search composed with a tag filter
(and (search "bycatch") (tag 'research))
Known deviations from the original spec #
Documented in more detail in design.md's Decisions/Open Questions and
inline in the relevant source files; summarized here:
- Quick-open matching is substring, not true fuzzy matching.
fuzzy_containsis a case-insensitive substring check, not a scored fuzzy algorithm (no subsequence matching, no ranking by match quality). The spec's own example scenario (trawl→trawler-design) happens to be a substring match, so it passes, but a query liketwdsnwould not matchtrawler-designthe way a real fuzzy matcher would. - Query-block syntax highlighting is a hand-rolled tokenizer
(
trawler::scheme_highlight), not tree-sitter as the spec names. See design.md D4 for the rationale — query blocks are short single expressions, not source files, so a real incremental-reparse tree-sitter grammar buys little here. gpui-componentwas dropped mid-project in favor of a customeditor::BlockEditorbuilt directly on GPUI's raw primitives. The original design leaned on it for the one-hot editor; its baked-in Enter/Tab/Backspace keybindings and a default-width bug fighting the outline's own semantics made it more friction than help. See design.md D3.- Cold start at large scale exceeds the original ~1s budget. Spike S3's
cold-start number (Loro doc load + a stand-in index rebuild) came in at
~667ms at 100k blocks, but that stand-in didn't account for the real
GraphIndex::rebuild(which parses references out of every block's content) or building the tantivy search index from scratch. Measuring with the actual code at 100k blocks: doc load ~0.2–0.9s (much slower uncompacted — see the graph-format note above),GraphIndex::rebuild~0.6–1.1s,SearchIndexbuild ~0.5s — roughly 1.5–2s total, over budget. The two rebuilds are independent of each other and currently run sequentially; running them concurrently (they don't share mutable state) is the obvious first optimization if this matters before a real corpus gets that large. Personal note-taking corpora are expected to stay well under 100k blocks for a long time, so this isn't blocking, but it's a real number, not the original hoped-for one.
Project layout #
crates/
trawler-core/ # block graph, Loro storage, indexes, Steel query engine,
# search — no UI dependencies, fully testable headless
trawler/ # GPUI app: outline editor, journal view, navigation,
# query block UI