# dbugs

A reimagining of [Rocicorp's zbugs](https://bugs.rocicorp.dev/p/zero) — the demo
app for their local-first sync engine, **Zero** — built as its architectural
antithesis: **Go + SQLite + [Datastar](https://data-star.dev)**, one binary, one
box, no client database.

### 🚀 Try it live: **[dbugs.yagni.club](https://dbugs.yagni.club)**

1,000,000 issues, full-text search, realtime multiplayer — on a $24/mo droplet.

[![screenshot](screenshot.png)](https://dbugs.yagni.club)

## The premise

A local-first sync engine buys UI latency by moving the data to the client:
sync the rows into the browser, run queries locally, and interactions are
instant because nothing crosses the network. The price is everything that
follows from the client holding the dataset — megabytes on first load, a client
query engine and sync protocol, auth questions a synced schema
[structurally can't answer](https://bugs.rocicorp.dev/p/zero/issue/3178), and
search that gets *disabled* once the dataset is too big to ship (the real zbugs
turns search off on its 240k-issue project).

dbugs is the opposite bet: **the browser holds nothing**. Each tab's filters,
sort, and open issue live on the server as a *search session*; the server
renders the visible window of that session to HTML and pushes it down one SSE
stream; Datastar morphs it into the DOM. Every interaction is a ~12-byte
command POST that mutates the session and lets the projection refresh.

The claim being tested: with a single node, SQLite, and the right architecture,
the server round trip is cheap enough that you don't need a sync engine to get
a performant UX —

- **Initial load is 2.3 KB** (brotli) instead of megabytes; data never looked
  at is never sent.
- **Filtered full-text search over 1M rows answers in ~10–20 ms** — facets are
  tokenized *into* the FTS index, so the feature the sync engine had to disable
  is one of the fastest paths here.
- **Writes commit in under a millisecond** through a single batching writer,
  and every other open tab sees them live over its stream — multiplayer for
  free, confirmed-not-optimistic.
- **1,000 concurrent streams sustain mixed writes at 44 ms push p50** (p99
  119 ms, zero errors) on one node.
- **Read auth is "what the template renders"** — row-level is a WHERE clause,
  cell-level is omitting a field from the HTML. No reader-dependent row types.

The honest trade: no offline, and the latency floor is a network round trip.
The answer is architectural (everything above) plus honest UX — optimistic
*cursor* state where it's safe, loading indicators that only surface when a
round trip is actually slow, and writes that confirm rather than pretend.

## Run

```bash
go run .            # seeds 10k fake issues into a throwaway temp-file SQLite, serves :8080
go run . -seed 1000000                    # the full-size demo (~614 MB on disk)
go run . -fresh=false -db file:dbugs.db   # persist to a named file instead
```

Tracing is built in and off by default; flip it on with standard OTel env vars
(spans become JSON lines on stdout — see the
[observability essay](slop-essays/observability.md)):

```bash
OTEL_TRACES_EXPORTER=console go run .
```

There's also a sanitized **live trace view** at `GET /traces` — public and on
by default, for education and transparency: the same spans, fat-morphed into a
page in realtime over the read stream as a Sentry-style waterfall (bars
positioned by time, color-coded by span category), with a live "N watching"
presence count. What makes it safe to expose is the sanitizer, not a gate:
every span passes a deny-by-default allow-list (no IPs, User-Agents, or issue
ids reach the browser) and trace ids are hashed with a per-process random salt
into anonymous handles. The view is rendered once per frame and fanned out to
all viewers, and is ephemeral — a bounded in-memory ring of the last 512 spans,
nothing persisted. No collector needed; it turns tracing on by itself. Disable
with `-trace-view=false`.

```bash
go run .                        # then open http://localhost:8080/traces
```

Load-test it with the purpose-built SSE harness (start the server with
`-writelimit=false`):

```bash
go run ./cmd/zload -readers 500 -writers 50 -rate 1 -dur 15s
```

## Architecture

CQRS over one SSE stream — commands mutate a server-side session, the read
model is rendered HTML:

```
┌──────────┐   command POST (~12 bytes)          ┌─────────────────────────┐
│ Browser  │  ───────────────────────────────►   │ mutate SessionState     │
│          │   /cmd/search /cmd/toggle-label     │ (the write model)       │
│ Datastar │   /cmd/select  /cmd/sort  /cmd/more │                         │
│          │                                     │ notify() ───┐           │
│          │   ◄─────────────────────────────    │             ▼           │
│  morph   │   ONE long-lived SSE stream         │ recompute projection    │
│  the DOM │   (the read model, brotli'd)        │ from SQLite, push morph │
└──────────┘                                     └─────────────────────────┘
```

State lives where it can be queried, authorized, and cached:

```
        browser (no data)                    server (all state)
┌───────────────────────────┐    ┌────────────────────────────────────────┐
│ rendered HTML              │    │ SessionState (per tab: query, filters, │
│ a handful of signals:      │    │   sort, window, selected issue)        │
│  · cursor position         │    │ SQLite (WAL):                          │
│  · form drafts             │    │   issues / labels / comments           │
│  · transient busy flags    │    │   FTS5(title, description, facets)     │
└───────────────────────────┘    └────────────────────────────────────────┘
```

The write path serializes through one connection and fans out by marking
sessions dirty — never by doing work inline:

```
POST /cmd/create ─┐
POST /cmd/comment ┼─► queue ─► ONE writer goroutine ─► BEGIN … SAVEPOINT/op … COMMIT
POST /cmd/edit ───┘            (a comment storm = one fsync)        │
                                                                    ▼
                                       dataVersion++ (only if the issue set changed
                                                      → invalidates count/list caches)
                                                                    │
                  sessions marked dirty (comments: only watchers) ◄─┘
                                  │
                     frame ticker (16 ms) wakes dirty sessions
                                  │
        per stream: render regions ─ count/list caches are SHARED across
        sessions (singleflight, 99% hit) ─► skip-unchanged dedup ─► brotli SSE
```

Reads scale because renders are shared; writes scale because they batch; idle
connections cost nothing because nothing happens until something is dirty.

| file | role |
|---|---|
| `main.go` | routing, the long-lived `/stream` (brotli SSE), the `/cmd/*` commands, security middleware, and the projection/diff loop |
| `db.go` | schema, deterministic fake seed, migrations, read-side queries, the faceted FTS index |
| `writer.go` | single-goroutine serialized write path (batched savepoints, version bump) |
| `session.go` | `SessionState` (write model) + per-session pub/sub hub + write rate-limit |
| `render.go` | view models + `html/template` fragments → strings for morphing |
| `countcache.go` / `listcache.go` | version-invalidated, singleflight-backed count and rendered-list caches |
| `auth.go` | GitHub OAuth + HMAC-signed stateless auth cookie |
| `otel.go` | OpenTelemetry setup: console (journald) or OTLP exporter, env-var gated |
| `templates/` | `index.html` shell + `fragments.html` (facets / list / detail) |
| `static/` | `app.css` + self-hosted `datastar.js` |
| `cmd/zload/` | load/QA harness (readers + writers + write→visible latency probe) |
| `deploy/` | systemd unit + setup/deploy scripts for a single-box deployment ([runbook](deploy/README.md)) |

## The long version

Everything measured and learned along the way — the wire-cost comparison
against the real Zero demo, scaling SQLite to 1M rows, the fan-out ceilings and
how each one moved, brotli memory dials, the deleted query planner, tracing
with no collector — lives in **[slop-essays/](slop-essays/README.md)**.