Waystation #

Waystation ingests events from the open web — webhook pushes and RSS/Atom feeds — and records each one into an Automerge document. Documents sync between peers with Subduction, so the ingest station is just another peer: replicas converge offline, and there is no canonical server to take down.

                    ┌──────────────────────────────┐
webhook POST ──────▶│  waystationer                │
                    │  (serve · poll · sync)       │
RSS/Atom poll ─────▶└──────────────┬───────────────┘
                                   │ normalized Items
                                   ▼
                    ┌──────────────────────────────┐
                    │  waystation_core (pure)      │
                    └──────────────┬───────────────┘
                                   ▼
                    ┌──────────────────────────────┐
                    │  waystation_doc (Automerge)  │
                    └──────────────┬───────────────┘
                                   │ changes
                                   ▼
                    ┌──────────────────────────────┐
                    │  Subduction sync             │◀──▶ peers
                    └──────────────────────────────┘

Decision logic is pure; I/O lives in the agent. The core owns the vocabulary and rules (ingest decisions, identity, routing); the waystationer binary owns the clocks, sockets, and storage.

Two properties, not two promises #

The architecture rests on two universally quantified claims, so both are stated as properties rather than examples:

• Merges never lose an accepted item. A station that committed an item still holds it after syncing with a peer.
• Replay is free. Re-running the durable queue moves no document heads.

The first one failed. Two stations recording different content for one identity — opened on one, closed on the other — each lost a side, silently, including the recording station's own payload. The cause was nested Automerge objects: put_object mints an object at a key, concurrent stations mint rivals, and the merge discards one subtree whole. The schema is now flat scalars, which have no subtree to discard — see document topology.

Libraries implement the protocol; agents that practice it are waystationers.

Development #

With Nix (flakes enabled):

nix develop   # rust 1.91, cargo-* utilities, command menu
menu          # list dev-shell commands (rust:*)

Or bring your own toolchain pinned by rust-toolchain.toml:

cargo test --workspace --all-features
cargo clippy --workspace --all-features --all-targets

CI mirrors the dev shell as runnable apps:

nix run .#ci   # fmt, clippy, test, cargo-deny

Design notes #

• Pure decision logic, I/O at the edges. Ingest decisions, identity hashing, and routing are pure functions in waystation_core; the waystationer agent owns the clocks, sockets, and storage (ADR-0015).
• Parse, don't validate. Origins carry structured addressing (endpoint names, feed URLs) rather than loose strings to be re-validated downstream.
• Records, not interpretations. Ingest stores the origin payload verbatim alongside normalized metadata; interpretation happens in documents, where it can be revised.
• Flat documents. Every value in a partition document is a root-level scalar. Nested maps read better and lose data under concurrent writes (ADR-0017).
• Paths name documents. A partition document's identity is a pure function of its path, so two stations that have never met mint the same document. No registry, no claims, no origin to elect (ADR-0018).

See design/ for the design documents (starting with document topology).

License #

Licensed under either of

• Apache License, Version 2.0 (LICENSE-APACHE)
• MIT license (LICENSE-MIT)

at your option.