diff --git a/docs/prototype/architecture.md b/docs/prototype/architecture.md new file mode 100644 index 0000000..4efd803 --- /dev/null +++ b/docs/prototype/architecture.md @@ -0,0 +1,148 @@ +# Verso: Architecture, First Pass + +> A working document. The shape of the system, not yet the spec. This document picks the bones; the data model and lexicon documents that follow pick the muscle. +> +> This is the third pass on the sync engine question. The first pass picked Zero. The second pass found Zero couldn't do offline writes and switched to LiveStore plus a custom Gleam sync backend. This pass commits to building the sync engine ourselves, on top of wa-sqlite, with our own event log, materializer, reactive query layer, and sync protocol. The decision trail is preserved in `02-sync-engine.md`. **The sync layer is an area of active study, not a settled specification.** + +--- + +## Premise + +The architecture exists to serve the commitments from the vision document: local-first behavior, instant interaction, intentional logging, real data ownership, beautiful native-feeling clients, content types that are honestly distinct, and longevity beyond any one maintainer. Every choice below is a means to one of those ends. Where two choices serve the vision equally well, the simpler one wins. Where a choice is fashionable but does not serve the vision, it is not made. + +There is one further commitment that emerged through the sync engine reconsideration and now belongs in the foundation: **we build what we want to understand, and consume what we don't**. The sync layer is something we want to understand — deeply, from first principles, in the Muratori tradition of working from the studs out. The browser's WebAssembly SQLite implementation is something we are content to consume. Drawing that line clearly is what makes the scope tractable. + +This is the first pass on the architecture as a whole. It is meant to be argued with. + +## The sync engine: built, not borrowed + +Verso runs on a custom-built sync engine. The full reasoning for moving past LiveStore to a from-scratch build lives in the sync engine document, but the short version is this: every off-the-shelf sync engine in the Postgres-friendly, offline-first lane is alpha-stage software whose maintenance shape doesn't match Verso's commitments. Zero refuses offline writes by design. LiveStore supports them well but its Solid adapter is officially "experimental and unstable" with the project explicitly asking for community maintainers, while the React adapter receives the meaningful API work. PowerSync is more entangled with its SaaS than its open-source surface admits. Building our own — at the granularity we're choosing — is more work than adopting any of these, and is also the work we most want to do. + +The boundary we draw: we build the layers where the design is interesting and the result reflects our taste. We consume the layers where the work is platform-specific accumulated engineering that someone else has already done well. + +We build: + +- **The event log.** Local persistence of immutable events, with ordering, integrity checks, and durability across browser sessions. This is the source of truth on every client. +- **The materializer.** The system that turns events into queryable SQL state. Each event has a materializer function that maps it to one or more SQL mutations against the local SQLite database. Re-running materializers from the event log produces identical state — this is what makes schema migration and replay tractable. +- **The reactive query layer.** Signal-based, designed to compose with Solid's reactivity primitives directly. A query is a live thing that updates synchronously when its underlying tables change. This is where the "instant" promise gets paid. +- **The sync protocol.** Push/pull/rebase between client and server, modeled on Git's mental model and on LiveStore's design. Local events are persisted before any sync attempt; pulled remote events are integrated through rebase before push retry; conflicts default to last-write-wins with hooks for custom resolution where the data type warrants it. +- **The Gleam server.** Event store with total ordering authority, notification fan-out to subscribed clients, and the durable Postgres backing. This is the part of the stack the user originally wanted to own, and it remains the part we are most enthusiastic about. + +We consume: + +- **wa-sqlite** ([rhashimoto/wa-sqlite](https://github.com/rhashimoto/wa-sqlite)). The WebAssembly SQLite implementation with browser storage VFS. This is years of platform-specific engineering — OPFS, IndexedDB, the cross-tab leader election story — that we are explicitly not relitigating. We use it directly, not through LiveStore's fork. +- **Solid's reactivity primitives.** `createSignal`, `createEffect`, `createMemo`. We compose with them; we do not reinvent them. +- **Postgres.** With WAL logical replication if and when we want it, but mostly just as the canonical event store on the server side. + +This is the honest scope. The build list is months of careful work, not weeks. But every hour is spent on a part of the stack we want to understand from the inside, and the result is a sync engine shaped exactly to Verso's needs rather than to a vendor's roadmap. + +The sync engine is, as of this document, an area of active study. The shape above is what we expect to build. The specifics — the on-wire protocol, the materializer interface, the conflict resolution hooks, the leader election story across tabs, the recovery semantics on disk corruption, the testing harness for distributed replay — are all unresolved. They will get their own design documents as the work proceeds. What follows in the rest of this architecture is stable enough to design around even while the sync layer remains in flux, because the rest of the architecture only depends on the sync layer through a small surface: events go in, materialized state comes out, queries are reactive over that state. + +## The language question + +With the sync engine being something we build, the language question collapses cleanly. The frontend and the sync client run in TypeScript because that's where Solid lives and where wa-sqlite is consumed. The backend is Gleam, end to end: the sync server, the AT Protocol firehose consumer, the RSS feed generator, the CSV importer, the metadata enrichment pipeline, the scheduled jobs. + +The biggest single workload outside the client runtime is the AT Protocol firehose consumer. That is a long-running WebSocket connection that ingests events from one or more PDSes (or a relay), validates them against our lexicons, and writes them to Postgres. It needs supervision, fault tolerance, backpressure handling, and the ability to crash and restart cleanly without losing work. This is exactly the workload BEAM was designed for, and Gleam gives us BEAM with type safety. The sync server is a similarly shaped workload — a long-running service maintaining many websocket connections, each subscribed to an event stream — and it lives naturally in the same runtime. + +The Gleam web ecosystem is now production-ready in the relevant sense. Wisp 2.2 ships in early 2026, runs on Mist, has type-safe routing and middleware, and benchmarks competitively. Pog handles Postgres connection pooling. Squirrel does typed SQL queries against the real schema. Crucially, Gleam is already in the user's stable language set, which means there is no ramp-up tax. + +Rust remains the right answer for a single specific workload that may emerge later: heavy CSV ingestion of fifty-thousand-book Goodreads exports, image processing for covers, or any pipeline that turns out to be CPU-bound. If and when that workload appears, we add a small Rust binary invoked as a worker. We do not preemptively introduce Rust into the system on the chance that it will be needed. + +Odin remains in the user's life as the language for Sylt. It does not appear in this stack. + +## Frontend + +Solid, with Vite as the build tool. The reactive query layer of our sync engine is being designed to compose directly with Solid's signal primitives, which is one of the practical reasons the build-our-own decision is more attractive than it might otherwise seem — we're not adapting around someone else's React-first runtime; we're building the runtime to fit Solid natively from day one. + +For the design system: Kobalte. It provides unstyled, accessible primitives for Solid in the same spirit as Radix for React or Headless UI. Pairing it with our own component layer gives us full visual control without re-implementing focus management, ARIA semantics, or keyboard navigation for every menu and dialog. Accessibility is one of the documented complaints with both Goodreads and StoryGraph, and starting from a properly built primitive layer is the cheapest way to ship a more accessible product than the incumbents from day one. + +For styling: UnoCSS, not raw CSS. The honest case for raw CSS is that we control everything; the honest case against it is that we are a small team trying to ship craft at speed, and atomic CSS with a strong design-token foundation is faster to build with and easier to keep consistent. UnoCSS has the additional advantage of being able to express the same atomic system as Tailwind without locking us into Tailwind's particular toolchain. The design tokens — colors, spacing, type scale, motion — live in one place, expressed once, and consumed by every component. + +The mobile story is web-first for the MVP, with Capacitor or Tauri Mobile as a credible path to native shells once the web app is solid. Native iOS and Android become a serious consideration the moment the web app's craft is in place — but not before. Building three apps in parallel from day one is how a small team fails. Building one beautifully and then porting the experience is how it succeeds. + +## Database and data shape + +Postgres for the durable event store on the server side, fed by the Gleam sync server. Local SQLite on each client (via wa-sqlite in the browser) holds the materialized view. The MVP runs a managed Postgres on Railway. + +Pog gives us a connection pool in Gleam. Squirrel gives us typed queries from real SQL. The schema lives in a `migrations/` directory of plain SQL files, run with a small migration tool. + +The data model is the single most consequential design choice the project will make, and the next document picks it specifically. The shape of it, foreshadowed here so the architecture can hang together: + +Events are uniform across content types. The vocabulary is `EngagementStarted`, `ProgressLogged`, `EngagementFinished`, `EngagementAbandoned`, `Rated`, `NoteAdded`, `Tagged`, and a handful of others. These verbs apply equally to a book, a podcast, a film, or a textbook. + +Content types are distinct. There is a `book_work` and a `book_edition` schema, with their own metadata fields and their own metadata sources (OpenLibrary, ISBNdb, Google Books, Hardcover's GraphQL). When manga ships in v2 there will be a `manga_work` and a `manga_volume` schema, with their own sources (AniList, MangaUpdates). When film ships there will be a `film_work` schema with TMDB. A book is not a podcast is not a film, and Verso treats this as an honest fact rather than a problem to abstract. + +Engagements bind events to works. An engagement is "this user, this work, this read-through" — a row that accumulates events over its lifetime. A re-read is a new engagement on the same work, with its own events. This is the data-model fix to one of StoryGraph's most-reported edition-state bugs and to Goodreads' inability to give different ratings to different reads. + +Tags and shelves hang off works (with an optional content-type filter), so a "favorites" shelf can hold books at v1 and grow naturally to hold films and manga later. + +The full schema is the next document. The point here is that the architecture supports it: events go through our sync engine, are persisted by the Gleam server, are replicated to Postgres, and are materialized into per-content-type SQLite tables on every client. New content types are additive — they ship a schema, a metadata source, and a UI; the events already exist. + +**Events are the durable artifact.** This bears emphasis. The sync engine, the materializers, the client runtime, the server implementation — all of these are software we wrote and may rewrite. The event log is what survives any of those rewrites unchanged. If we discover a year in that some structural decision in the sync engine was wrong, we replay the event log into a corrected runtime. If we decide to migrate from Postgres to something else on the server, we replay events. The events are the contract; everything else is implementation. + +## AT Protocol layer + +The AT Protocol does three things for Verso, and it is important to be clear that it does not replace our sync engine — it sits beside it. + +First, identity. A user signs in with their AT Protocol handle (their `did:plc:...` resolved through PLC, or `did:web:...` resolved through DNS). The handle becomes their stable identity inside the app. We do not run our own user database with passwords; we delegate that to whichever PDS the user already trusts, the same way Tangled does for code repositories. This solves a problem the vision document raised explicitly — that an indie SaaS holding all your data is not actually a meaningful improvement on Amazon holding all your data — and it solves it through delegation rather than new infrastructure on our part. + +Second, public records. When a user marks something as engaged-with, that event is private to them by default and lives in their local event log (synced through our backend, replicated to Postgres). When they choose to make a review, a finished-engagement announcement, or a public shelf visible to the wider network, we publish a record to their PDS under one of our published lexicons (something like `app.verso.book.read` and `app.verso.book.review` for v1, with parallel namespaces for other content types as they ship). The canonical copy of that public record is in the user's PDS, not in our database. This makes Verso a participant in the AT Protocol data network rather than a walled garden that happens to use AT Protocol for login. + +Third, social. Eventually — not in v1 — friends, follows, and buddy reads can lean on the AT Protocol social graph the user has already built on Bluesky and elsewhere, rather than asking them to rebuild it inside Verso. The "we won't fight Goodreads on social" stance from the vision document survives intact: we are not building a social network, we are participating in one that already exists. RSS sits next to this layer, providing a more universal subscription path for users and tools that do not speak AT Protocol. Each user gets a stable RSS feed of their public engagement activity. Both formats are first-class because both serve the data-portability commitment. + +The Gleam services own this layer. The firehose consumer subscribes to PDS or relay events, filters for our lexicons and for users we care about, and writes the results into Postgres. The publishing path goes the other direction: when a user makes something public, the Gleam service signs and writes a record to their PDS via the standard AT Protocol XRPC calls. Lexicons are versioned, published at our domain, and treated as a public contract once shipped — breaking changes mean a new lexicon, not a mutated one. + +The AT Protocol model maps unusually well to our event-sourced model. Both systems treat user data as an ordered log of immutable signed records; the difference is mostly where the records are stored and which ones are visible to whom. Publishing a private internal event as a public AT Protocol record is a transformation, not a translation. + +## Source hosting and deployment + +Tangled hosts the source. The vision document already said this and the architecture defers to it: the repository's social and ownership model is part of what we are building, not a deployment detail. Practically, this means a Tangled knot (managed by Tangled for the MVP, possibly self-hosted later) and pull-request flow through their AppView. CI runs through Tangled's "spindle" system if it is mature enough by the time we need it; otherwise we run CI ourselves on a small box. + +Railway hosts the running services for the MVP. It runs Postgres, the Gleam sync server, the Gleam services for AT Protocol firehose and RSS, and the static frontend (or the frontend ships to a CDN like Cloudflare Pages and only the API services run on Railway). Railway is a bridge, not a destination. The whole stack should be portable to any container host without rewriting anything more substantial than environment variables. + +The MVP topology is small: one Postgres instance, one Gleam service that hosts the sync server plus the AT Protocol firehose plus RSS plus ingestion jobs (split into separate services later if the workloads need it), and the static frontend. Three things. Small enough to reason about, large enough to mean what we want it to mean. + +## What this architecture is deliberately not doing + +It is not a microservices architecture. The Gleam service stays as one service until a real seam appears that justifies splitting it. Premature service-splitting is a tax we do not need to pay. + +It is not multi-tenant in any complex sense. Every user's data lives in shared Postgres tables with row-level filtering, keyed by AT Protocol DID. We do not run a database per user, a schema per user, or anything fancier. The local-first nature of the system means each user's working set is isolated on their own device anyway; the server-side multi-tenancy story can stay simple. + +It does not include CRDTs. Our sync model is event-sourced with rebase-on-pull and last-write-wins as the default merge strategy. This is correct for the dominant case of one person editing their own engagement log. The few cases that need richer resolution — collaborative shelves, buddy reads — get custom merge functions written for them specifically, which is a perfectly reasonable amount of work for a perfectly small set of cases. CRDTs are a fascinating piece of technology and the wrong tool for a system whose model is "the user is editing their own log." + +It does not pre-pay for scale we do not have. The architecture should run cleanly for the first ten thousand users on a single Postgres instance and a single Gleam service. When that stops being true, we revisit. We do not introduce Kafka, Kubernetes, or a service mesh in v1. + +It does not auto-track. There is no scrobbler service in this architecture, no streaming-platform integration, no Kindle sync daemon. Logging is intentional, per the vision document, and the absence of these services is a design choice. + +## Open questions and risks + +The sync engine is the largest single risk in the architecture, by design. Building it ourselves means owning every bug, every edge case, every corruption-recovery scenario, every cross-tab leader-election awkwardness. The mitigation is the boundary: we consume wa-sqlite for the parts that are platform-specific, and we own the parts where the design is the interesting work. We also accept that "shipping" is further away than it would be with an off-the-shelf sync engine, and that this is part of the cost we are paying for the kind of project this is. + +Active study areas for the sync engine, in rough order of how soon they need answers: + +- The on-wire protocol between client and Gleam server (event envelope format, cursor semantics, batching, reconnection). +- The materializer interface (how event handlers register, how they declare which tables they touch, how replay-from-zero works). +- The reactive query layer's integration with Solid (how query subscriptions become signals, how invalidation propagates without re-running queries that don't depend on changed tables). +- Cross-tab coordination on a single device (which tab is the leader for sync, how state is shared, what happens when the leader closes). +- Disk corruption recovery on the local SQLite database (do we always replay from the event log, or do we trust the SQLite snapshot and only replay tail events). +- Conflict resolution beyond last-write-wins (where does it live in the API surface, what are the per-content-type hooks). + +None of these need to be resolved for this architecture document to hang together. They will be addressed in dedicated sync engine design documents as the work proceeds. What this document commits to is the **shape**: built not borrowed, wa-sqlite for the WASM SQLite layer, Gleam for the server, events as the durable artifact, Solid as the reactivity context. + +The metadata problem remains the hardest hidden problem in the category, and no architecture decision we have made addresses it. OpenLibrary is the most-permissive source and has known data-quality issues. Hardcover's GraphQL is good but is a single-vendor dependency. ISBNdb is paid. Google Books is paid past a low quota. We will need a metadata strategy that combines sources, deduplicates, and handles the edge cases — translations, audiobook ASINs, omnibus editions — before we have a real product. This is engineering work that does not show in the topology diagram and will quietly take months. Each new content type added in v2 brings a parallel metadata problem with its own sources and edge cases. + +The lexicon design is its own document. Naming the records we will publish — what counts as a public engagement event, what fields they contain, how they version — is a substantial design exercise that affects whether other AT Protocol apps will treat us as a serious citizen of the atmosphere or as a one-off. + +The native mobile question — Capacitor versus Tauri Mobile versus actually native — is unresolved. We can defer it until the web app is good. We should not defer it past the point where the web app is good, because mobile is where serious readers track most of their reading and a great web app on a phone is still a worse experience than a competent native one. The sync engine work should keep this in mind: any client-side code that assumes a browser context will need to be portable to a native shell later. + +## What comes next + +The next document picks the data model. The schema is where the design commits to specific stances on re-reads, editions, audiobook time, content-type separation, tag organization, and the rest of the things the incumbents got wrong. Designing the event vocabulary is the substantive part — the SQL tables fall out of it cleanly once the events are right. + +After that, the lexicon document — which AT Protocol records Verso publishes, and how they map onto our internal events — follows naturally because the work is half-done. + +Parallel to those, the sync engine begins its own series of design documents. The first one worth writing is the on-wire protocol, because everything else in the sync engine depends on what messages mean. After that, the materializer interface; after that, the reactive query layer; after that, cross-tab coordination. These can be written and refined while the data model and lexicon work proceeds. + +The first piece of code worth writing is small but real: a Gleam Wisp service that accepts a minimal sync protocol — append events to an ordered log, fetch events from a cursor, notify subscribed clients of new events — backed by Postgres. It is a focused, learnable, fun piece of software. Building it before writing the application code is the right order: it forces us to engage with the event model from the start, and it gives us a piece of working infrastructure we will use every day. + +This document picks the bones. The shape of the system is now legible enough to argue with. diff --git a/docs/prototype/fullstack-gleam.md b/docs/prototype/fullstack-gleam.md new file mode 100644 index 0000000..2887958 --- /dev/null +++ b/docs/prototype/fullstack-gleam.md @@ -0,0 +1,244 @@ +# Fullstack Gleam: A Minimalist Architecture Spec + +## Project Goals + +Build an end-to-end web application using Gleam as the only application language, targeting BEAM on the server and JavaScript on the client. Use HTML and CSS directly without a CSS framework or build pipeline beyond what Gleam itself requires. Be radically selective about dependencies, in the spirit of building from primitives rather than assembling pre-made layers. The architecture should be small enough to hold in one head, performant by virtue of doing less, and structured so that each piece can be replaced without rewriting the others. + +This is a learning and exploration project. The constraints are deliberate. + +--- + +## High-Level Architecture + +The system is a monorepo of three Gleam packages that share types and logic across the server/client boundary, plus a static asset directory and a SQLite database for materialized state. The runtime model treats the application as a reduction over an event log: state at any moment is a fold of all events to date, the view is a pure function of that state, and the only thing that causes a render is an event being dispatched. There is no virtual DOM, no signal graph, no auto-tracking reactive system. There is one update mechanism, and it is explicit. + +### Package Layout + +``` +project/ +├── shared/ # Targets both Erlang and JavaScript +│ ├── src/ +│ │ ├── shared/events.gleam # Event type, decoders, encoders +│ │ ├── shared/state.gleam # Application state types +│ │ ├── shared/reducer.gleam # apply(state, event) -> state +│ │ └── shared/validation.gleam # Pure validation functions +│ └── gleam.toml +├── server/ # Targets Erlang only +│ ├── src/ +│ │ ├── server.gleam # Entry point, mist + wisp setup +│ │ ├── server/router.gleam # Top-level path dispatch +│ │ ├── server/handlers/ # Per-route request handlers +│ │ ├── server/views/ # nakai view functions +│ │ ├── server/db.gleam # SQLite or Postgres access +│ │ └── server/sync.gleam # Event log endpoints +│ ├── priv/static/ # Served by wisp +│ └── gleam.toml +├── client/ # Targets JavaScript only +│ ├── src/ +│ │ ├── client.gleam # Entry point, mounts to DOM +│ │ ├── client/runtime.gleam # Stable-key renderer +│ │ ├── client/dispatch.gleam # Event dispatch + render scheduling +│ │ ├── client/sqlite.gleam # Browser SQLite bindings +│ │ ├── client/views/ # Client-side nakai-style views +│ │ └── client/sync.gleam # Event log sync to server +│ └── gleam.toml +└── static/ # Hand-written CSS, copied to priv/ +``` + +The `shared` package depends on nothing but `gleam_stdlib` and `gleam_json` so that it compiles to both targets cleanly. The `server` and `client` packages both depend on `shared` and add their target-specific dependencies. This division is the structural foundation everything else rests on. + +--- + +## The HTTP Layer + +The server is built on `mist` for the underlying HTTP/WebSocket transport and `wisp` for the request/response conventions on top of it. Wisp owns the request lifecycle, body reading, multipart parsing, cookie signing, CSRF token helpers, and static file serving from the `priv/` directory. It does not own routing; the router is a single function that pattern-matches on `wisp.path_segments(req)` and `req.method` and delegates to handler functions. For sub-trees of the URL space, the top-level router delegates to nested routers that pattern-match on the remaining segments. + +Middleware is composed using Gleam's `use` syntax. A typical handler begins with a small stack of `use <- wisp.log_request(req)`, `use <- wisp.rescue_crashes`, `use <- wisp.handle_head(req)`, and any application-level middleware (authentication, request ID generation) before the actual route logic. There is no registration step. The middleware pipeline is the call graph, readable top to bottom. + +Body handling defers entirely to wisp. Forms come back as decoded structures. File uploads go to temp files. Size limits are enforced before allocation. Cookie reading and writing uses wisp's three-tier model (plaintext, signed, signed-and-encrypted) with the secret key base configured at server boot. + +--- + +## The View Layer + +Server-rendered HTML is produced by `nakai`. View functions are ordinary Gleam functions that take data and return `nakai/html.Node` values, composed freely with `case` expressions, `list.map`, and function calls. There is no template language, no component lifecycle, no props validation beyond Gleam's type system. Layouts are functions that wrap content in a shell; partials are functions returning fragments. The seam to wisp is `view |> nakai.to_string_tree |> wisp.html_response(200)`. + +All text content is HTML-escaped by default. Raw HTML insertion is available through an explicit `UnsafeText` constructor, making the dangerous path visible at the call site. Document-level concerns (head injection from deep components) are handled by nakai's `Document` type, which lets a chart component declare its own stylesheet dependency without the top-level layout knowing about it. + +The same nakai code can be used on the client when the client package needs to produce HTML strings — for example, for hydrating an island or for producing the initial render that the client runtime walks. This is the practical payoff of the shared/server/client split: a single set of view functions runs in both environments. + +--- + +## The Event Log Model + +The application's canonical state is an append-only log of events. Events are typed Gleam values defined in `shared/events.gleam` with derived JSON encoders and decoders. The log is the source of truth; everything else is derived. State at any moment is the fold `list.fold(events, initial_state, apply)`, where `apply` is the pure reducer in `shared/reducer.gleam`. + +This model eliminates a class of bugs around "what is the current state?" because the answer is always "fold the log." It also makes server-client synchronization conceptually trivial: synchronization is the act of merging two logs and re-folding. Optimistic local updates are events appended locally before the server has confirmed them; server confirmation is another event in the log; rejection is a third. There is no separate code path for these cases at the view layer, because the view layer reads materialized state and does not care which events produced it. + +### Materialization + +Folding the entire log on every render is impractical past trivial sizes, so events are materialized into queryable state. On the client, this is a SQLite database running in the browser via `sql.js` or `wa-sqlite`, with bindings written through `plinth`. On the server, this is either the same SQLite file (via `sqlight`) or a Postgres database (via `pog`), depending on deployment shape. + +Materialization runs each event through a function that translates it into SQL writes. For example, an `ItemAdded(id, title)` event becomes `INSERT INTO items (id, title) VALUES (?, ?)`. The view layer queries SQLite directly and gets back ordinary Gleam values via `gleam/dynamic/decode`. There is no ORM and no query builder; SQL is written as strings. + +The initial implementation invalidates all queries on every committed event, re-runs the view function in full, and lets the renderer's stable-key walk absorb the cost of producing identical output where nothing changed. This is dramatically simpler than tracking query dependencies and remains performant up to surprisingly large UIs because the renderer does the minimum work in the steady state. A dependency-tracking layer that maps SQL tables to dependent queries can be added later if profiling demands it; this is the LiveStore-style "reactive SQLite" optimization, and it is an optimization, not an architectural requirement. + +--- + +## The Renderer: Stable-Key Direct Construction + +The client renderer walks the live DOM in lockstep with the calls a view function makes, deciding at each step whether the existing node matches what the call wants. There is no virtual DOM and no diff between two trees. There is one tree — the live DOM — and a forward walk that touches it only where reality differs from the description. + +### Core Mechanism + +The runtime maintains during a render pass a cursor over the DOM (current parent, current child position within that parent), a side table keyed by stable path holding event handlers and any other transient per-node state, and a scratch list of nodes scheduled for removal at the end of the pass. + +A primitive call like `ui.div(key, attrs, children)` performs the following. It examines the cursor's current position. If the node there is a `div` with the matching key, attribute deltas are applied in place. If the node does not match, a new `div` is inserted before the current node and the existing one is marked for removal. The runtime then descends: pushes the cursor, makes the new `div` the parent, resets child position to zero, runs the children block, pops back. At the end of the children block, any unconsumed siblings of the new parent are scheduled for removal. + +### Stable Keys + +Static structure (a header followed by a main followed by a footer) uses positional identity. The third call in a pass corresponds to the third node from the previous pass, because the call sequence does not change shape. Dynamic structure (a list of items where insertion in the middle is possible) requires explicit keys derived from the data, exactly as in any keyed reconciliation. Keys are consulted directly against the live DOM (typically via a `data-key` attribute or an internal map keyed by parent), not against a cached previous tree. + +### Event Handlers + +Event handlers cannot be compared for equality across passes, so the runtime always replaces them. Replacement is cheap because handlers are stored in the side table keyed by `(path, event_name)` and a single dispatcher is attached at mount time. The dispatcher resolves the handler by path when an event fires. Replacing a handler is a table write, not a `removeEventListener`/`addEventListener` pair. + +### API Sketch + +```gleam +// Primitive elements take a key, attributes, and children. +// Children are produced by a closure so use-syntax composes naturally. +pub fn div(key: Key, attrs: List(Attr), children: fn() -> Nil) -> Nil +pub fn button(key: Key, attrs: List(Attr), children: fn() -> Nil) -> Nil +pub fn text(content: String) -> Nil + +// Event handlers are attributes that close over a dispatch function. +pub fn on_click(handler: fn() -> Msg) -> Attr + +// A view is a function that, given state, performs a sequence of calls. +pub type View(state) = fn(state) -> Nil + +// Composition uses Gleam's use syntax for nesting. +fn item_card(item: Item) -> Nil { + use <- div(Key(item.id), [class("card")]) + text(item.title) + use <- button(Key("delete"), [on_click(fn() { Delete(item.id) })]) + text("Delete") +} +``` + +The closure-based children API is the Gleam-native way to express nesting, and it pairs cleanly with `use`. The renderer's job is to interpret these calls against the live DOM. + +### Performance Characteristics + +The steady-state cost of a render pass is one DOM node visit per node in the view, plus attribute equality checks. There is no allocation for matching subtrees and no garbage to collect. Mutations only happen where the description differs from reality. This is the same asymptotic profile as a fine-grained reactivity system, achieved through a different mechanism: instead of tracking dependencies to update only changed nodes, the renderer walks everything cheaply and updates only changed nodes. + +The expected size of the renderer is roughly two hundred lines of Gleam plus a handful of `plinth` calls into the DOM API. This is small enough to read in one sitting and to debug by stepping through. + +--- + +## Dispatch and the Render Loop + +The single entry point for state change is `dispatch(event)`. It performs four steps in order: appends the event to the local log, applies it through the reducer (running the SQL writes for materialization), schedules a render on the next animation frame if one is not already scheduled, and forwards the event to the sync engine for transmission to the server. + +There is no other way to cause a render. This is the explicit-rendering model, and it is the architectural commitment that keeps the system comprehensible. If a re-render is needed, an event was dispatched. If no event was dispatched, no re-render happens. Performance behavior is predictable because the trigger graph is trivial. + +The render scheduling uses `requestAnimationFrame` to coalesce multiple dispatches in the same tick into a single render pass. This handles the common case of a UI action producing multiple events (open a modal, focus an input, log an analytics event) without three render passes. + +--- + +## Server-Client Sync + +The sync engine treats events as the unit of replication. The client maintains a local event log and a "last synced sequence number." On dispatch, events are appended locally and immediately materialized for the optimistic view. A background task batches unsynced events and POSTs them to a sync endpoint on the server. The server validates each event against its authoritative state, appends accepted events to its log, and returns the canonical sequence numbers plus any rejection events. + +Rejection is itself an event. The reducer in `shared/reducer.gleam` knows how to handle a `Rejected(event_id, reason)` event by reverting whatever the optimistic event did. This keeps the model uniform — there is one update mechanism, not separate ones for happy and sad paths. + +The transport is a simple HTTP long-poll or WebSocket, depending on what feels appropriate. Both are available through `mist`. The sync protocol is intentionally not real-time-collaborative in the first pass; conflict resolution is "server wins, client reconciles." Operational transformation, CRDTs, and richer conflict models are deferred until the basic shape is working. + +--- + +## Data Flow Summary + +The system has one shape: + +A user action produces an event. The event is dispatched, which appends it to the local log, materializes it into local SQLite, and schedules a render. The renderer runs the view function, which queries SQLite for whatever it needs, and walks the live DOM applying minimal mutations. The event is also sent to the server, which validates and persists it, and either confirms or rejects it. Confirmation or rejection arrives as another event, which goes through the same dispatch path. + +Server-initiated changes (another user's actions, scheduled jobs, external systems) arrive via the sync channel as events into the local log and proceed identically. There is no special case at the view layer for "this came from elsewhere"; events are events. + +--- + +## Initial Page Load + +Initial page load is server-rendered through the same nakai view functions. The server materializes state for the requested page, runs the view function, and returns HTML. The client bundle loads, queries the server for the current event log sequence number, and adopts the rendered HTML as its initial DOM. The renderer's first pass is a no-op walk that confirms the live DOM matches what the view function produces from current state, after which subsequent passes update incrementally. + +This is structurally similar to hydration in other frameworks, but conceptually simpler because the renderer's normal operation already handles "the DOM exists; reconcile against it." There is no separate hydration code path. The first pass is just a pass where every node happens to already be correct. + +Resumability in the Qwik sense — deferring handler attachment until interaction — is a future optimization. It would require serializing dispatch information into the HTML and lazy-loading the relevant handler closures. The architecture does not preclude this, but it is not in scope for the initial build. + +--- + +## CSS + +CSS is hand-written in the `static/` directory, copied to `priv/static/` at build time, and served by wisp. The strategy uses cascade layers (`@layer reset, base, components, utilities`) to control specificity, container queries for component-local responsiveness, custom properties for design tokens, and `@property` registrations where animation of computed values is needed. There is no preprocessor, no PostCSS, no Tailwind. Modern CSS is sufficient. + +Component-local styles can be co-located with their nakai view functions by emitting `