xenolandscapes.com / xenomorphic
The agentic engineering control plane for the posthuman future
Star
Fork 0
Watch
xenomorphic notes daemon-ui-split.md
25 kB
Markdown
at main

Xenomorphic daemon / UI split — design #

Status: design agreed by user 2026-09-08. Not yet implemented.

Related notes:

  • notes/daemon-contract.md — normative phase 1 wire contract. Implementers code against that file; this one holds the rationale.
  • notes/daemon-remote-tailscale.md — deferred remote/Tailscale design plus a pickup checklist.

Decisions taken #

  1. Daemon is a detached process the app spawns itself. No service registration, no launchd, no systemd unit. No install step and no permission prompt. Survives app quit, app crash, app relaunch. Dies at logout or reboot. Accepted.
  2. No on-disk byte log. In-memory only. After daemon death the byte history is gone. Recovery is respawn: PaneRecord.restart + restartArgv + spawnCwd relaunch the programs, panes come back with empty screens. Accepted.
  3. The daemon owns the session structure. workspaces → tabs → split tree → panes live in the daemon, because a client cannot know which panes to request or how to lay them out otherwise.
  4. Daemon classes mirror the current QObjects 1:1 in both data and behavior. See "AppModel relocation".
  5. WebSockets only, one transport. QWebSocketServer, local now and remote later, so remote stays additive. See "Protocol".
  6. Transparent auth (local). Per-boot token file (0600) sent in the handshake header. No prompts.
  7. Log trim policy settled. Trim only at parser-ground record boundaries; 10 MB cap; below-floor attach returns the real floor.
  8. Windows support revoked (2026-09-08). POSIX only: macOS + Linux. The ConPTY branch was already dead code, since flake.nix:10 lists only x86_64-linux, aarch64-linux, aarch64-darwin. Leave the #ifdef _WIN32 branches in place; removing them is a separate cleanup decision.
  9. PtyClient mirrors Pty's shape; no shared base class (2026-09-08). The two classes never coexist in one process, so no interface is needed. Supersedes the earlier "extract an interface" plan.
  10. PtyClient::close() means detach, never kill (2026-09-08). The single most load-bearing change in the split. Today the kill is implicit in ~TerminalQuickItem, which is exactly why shells die at quit. If close() forwarded to a daemon kill, the daemon would buy nothing. Killing becomes an explicit /ctl message sent only from the user-initiated close path. Normative detail in notes/daemon-contract.md.
  11. Remote/Tailscale deferred (2026-09-08). Daemon binds 127.0.0.1 only. Local-only auth. The settled remote design lives in notes/daemon-remote-tailscale.md, kept cheap by the WebSockets-only choice: picking it up adds a listener plus an authorization check, not a new transport or codec.

Central principle #

Terminal state is a pure function of the PTY byte stream plus the resize history. Therefore the daemon needs no terminal semantics. It stores bytes and replays them. Every client keeps its own local GhosttyTerminal, renderer, selection, kitty graphics, and glyph atlas, all unchanged.

This is why the byte log beats ghostty's snapshot API (ghostty_snapshot_encode_alloc, snapshot.h:284-517):

  • snapshot.h defines no kitty graphics record tag, so a 256 MB kitty store (TerminalQuickItem.cpp:294-295) would be lost.
  • encode requires GHOSTTY_TERMINAL_OPT_CONTINUATION_MAX_BYTES set before the input that produced a mid-sequence state (snapshot.h:267-269, terminal.h:1797-1799). Not set today, so encode would return GHOSTTY_INVALID_VALUE on any pane caught mid escape sequence.
  • format v1 has no compatibility guarantee (snapshot.h:112-113).
  • replaying the real stream costs nothing extra and loses nothing.

Why shells die today #

TerminalQuickItem owns m_pty as a QObject child (TerminalQuickItem.cpp:63). App quit destroys the item, ~Pty closes m_masterFd. Closing the master makes the kernel SIGHUP the slave's foreground process group and the shell exits. Whoever holds the master fd decides whether shells live.

Note: the source contains no setsid(), but forkpty() (Pty.cpp:516) performs it in the child per its documented contract. The shell is already a session leader with the pty slave as controlling terminal. [inference from documented forkpty(3) behavior, libc source not read]

Corollary that matters for the daemon: because the shell is already a session leader with the pty as its controlling terminal, moving the master fd to another process does not change the shell's session or job control. The daemon can spawn panes exactly as the app does today and tcgetpgrp (Pty.h:68-72) keeps working.

Process layout #

xenomorphicd   QtCore + QtNetwork + QtWebSockets only. No QtGui, no QtQuick,
               no libghostty-vt.
               owns: Pty instances, master fds, shell children, pids,
                     per-pane ordered byte log, session structure,
                     SessionStore (session.json)
xenomorphic    unchanged renderer. owns: GhosttyTerminal, GhosttyRenderState,
               selection, kitty, glyph atlas, shaders, QML, controllers,
               XenoSettings
future client   same codec, same WebSocket transport, remote URL. Deferred:
                see notes/daemon-remote-tailscale.md

Daemon links no ghostty. Verified feasible: ghostty is a pure C library and the daemon does not need it. Daemon links qtwebsockets.

Protocol (settled 2026-09-08) #

Connection topology #

One control connection per client, always open. Plus one data connection per pane, opened lazily: a client streams only the panes it actually displays. A phone showing one pane opens one data connection. Desktop with four visible panes opens four. Background panes are never attached, so there is no cold-replay storm and no head-of-line blocking between panes.

Attach and detach are connect and close. No protocol churn on tab switch or split.

Verified constraint: QWebSocketServer is TCP-only #

From qtwebsockets-6.11.2/lib/QtWebSockets.framework/Versions/A/Headers/qwebsocketserver.h:

  • listen(const QHostAddress&, quint16 port) (line 50). Host address and port only.
  • handleConnection(QTcpSocket *socket) (line 112). Takes a QTcpSocket, so QLocalSocket is not usable.
  • setSocketDescriptor(qintptr) (line 77) exists, but QtWebSockets wraps the fd in a QTcpSocket, which is AF_INET only. An AF_UNIX fd will not work.

QtWebSockets 6.11.2 is in nixpkgs and matches the project's Qt 6.11.1, so availability is not the issue. Choosing WebSockets forces TCP loopback locally and loses Unix domain sockets.

Decision (revised 2026-09-08): WebSockets only, one transport #

Centralize on QWebSocketServer for both local and remote. No QLocalSocket path, no second adapter.

What this costs and why it is acceptable: QWebSocketServer is TCP-only (verified above), so local clients connect over TCP loopback instead of a Unix socket, and the free 0600 filesystem gate is gone. Authentication must therefore be explicit. See "Transparent authentication" below.

What this buys:

  • One transport adapter, one codec, one connection lifecycle to test.
  • Frame boundaries come free. WebSocket frames are already delimited, so the control plane drops newline framing entirely: one JSON object per text frame. The DevDriver's newline delimiter existed only because TCP is a byte stream (DevDriver.cpp:27-40); WebSocket removes that need.
  • Forward-looking: when remote lands, it runs the identical code path. A remote client is not a special case, it is the same client pointed at a different URL. This is the main reason to accept WebSockets locally even though remote is deferred.
  • Keepalive ping and close codes come from the library. TLS only matters for the deferred remote listener.

Performance note that favors this choice on the hot path: RFC 6455 requires client-to-server frames to be masked and forbids servers from masking. The XOR masking cost therefore lands on keystrokes, which are tiny. PTY output, the high-volume direction, travels server-to-client unmasked.

Verified QWebSocket API surface relied upon #

From qtwebsockets-6.11.2/.../Headers/qwebsocket.h:

  • open(const QNetworkRequest&) (line 108) and request() (line 71): handshake headers work on both client and server side, so the token can ride the WebSocket handshake natively. No in-band auth message needed.
  • peerAddress() (line 51): the peer IP. Unused locally, needed by the deferred Tailscale whois gate.
  • setPauseMode / pauseMode / resume (lines 50, 64, 63) and bytesToWrite() (line 89): real backpressure. This is what keeps a slow client from blocking the pty reader thread.
  • sendBinaryMessage / sendTextMessage (lines 79-78), binaryMessageReceived / textMessageReceived (lines 130-129): the two planes map directly onto binary and text frames.
  • setOutgoingFrameSize (line 98), setMaxAllowedIncomingFrameSize (line 91), setMaxAllowedIncomingMessageSize (line 93): frame and message limits are configurable, so the 10 MB log cap and 16 KB pty chunks can be sized deliberately.
  • close(CloseCode, reason) (lines 103-104) with the full code set in qwebsocketprotocol.h:35-49: CloseCodePolicyViolated (1008) for auth failure, CloseCodeTooMuchData (1009) for a client that exceeds limits, CloseCodeBadOperation (1011) for protocol violations.

Close codes are worth using properly. They let a client distinguish "you are not authorized" from "you fell behind and must re-seek" without an extra in-band error message.

Transparent authentication #

Local-only for now. One mechanism, automatic, no prompt.

Local clients: per-boot token file.

The daemon generates a random token at start and writes it with its listen port to ${XDG_CONFIG_HOME:-$HOME/.config}/xenomorphic/daemon.json at mode 0600. Clients read that file, then send the token in the handshake.

Rejected alternatives and why:

  • Peer credentials (SO_PEERCRED): does not work at all here. It is a Unix-domain-socket facility, and WebSocket is TCP. Dead on arrival.
  • OS keychain: prompts on macOS, and needs three platform integrations. Contradicts "transparent".
  • Static long-lived token: a token that never rotates leaks permanently. Per-boot rotation bounds the damage.

Security analysis, stated plainly. With a Unix socket at 0600 the kernel gates the connection by file permission. With a token file at 0600 the daemon gates it after accept. The trust boundary is the same in both cases: can this process read a 0600 file in your home directory? A process running as your uid can already read your files and spawn your shells, so it gains nothing. A process running as another uid can neither connect (port is bound to loopback only, and it cannot read the token) nor read the token. The daemon must bind 127.0.0.1 for local, never 0.0.0.0.

Optimization: the app spawns the daemon, so the first client can receive the token over the spawn pipe or argv and skip the file read entirely. The file is still required, because a relaunched app must discover an already-running daemon.

Remote clients: deferred.

The daemon binds 127.0.0.1 only and never 0.0.0.0. There is no remote listener, so no remote authorization path is needed now.

The settled remote design (Tailscale whois on accept, macOS localapi socket paths, App Store TCP+token fallback, TLS via Tailscale node cert, fail closed) is recorded in notes/daemon-remote-tailscale.md with a pickup checklist. It stays cheap because WebSockets is already the only transport: picking it up adds a listener plus an authorization gate on accept, not a new transport or codec.

Control plane #

One JSON object per text frame. No newline delimiter: WebSocket frames are already delimited, so the DevDriver's newline framing (DevDriver.cpp:27-40) is unnecessary here and is dropped.

Authentication rides the handshake via QNetworkRequest headers (qwebsocket.h:108), not an in-band auth message. A client that is not authorized never gets a usable control connection; it is closed with CloseCodePolicyViolated during or immediately after the handshake.

Client → daemon: spawn(spec) -> paneId, kill(paneId), resize(paneId,cols,rows) (controlling client only), structure(), mutate(op).

Daemon → client: exit(paneId,code), cwd(paneId,path), structureChanged(...) notifications.

Data plane #

A data connection opens with the pane and start sequence in the handshake URL, so attach needs no in-band message either:

ws://127.0.0.1:PORT/pane/p7?fromSeq=12000

(The deferred remote form is wss://host/pane/p7?fromSeq=12000; only the scheme and the authorization mechanism differ. See notes/daemon-remote-tailscale.md.)

The daemon answers with one text ack (confirming the real floor if fromSeq was below it), then the connection carries only binary frames for the rest of its life.

Binary frame fields: stream id (u32), seq, tag, payload length, payload.

Stream id is a compact u32 the daemon assigns at attach time, not the pane's string id. A string in every frame would be wasteful. The u32 is redundant on a per-pane connection today. Pay the four bytes anyway: it makes a future multiplexed single-connection transport a pure transport change with no codec change.

Tags: BYTES, RESIZE, EXIT. That is the whole data plane.

Sizing: setOutgoingFrameSize (qwebsocket.h:98) should be tuned to the pty chunk size, which is 16 KB today (Pty reader thread buffer). setMaxAllowedIncomingFrameSize/setMaxAllowedIncomingMessageSize (qwebsocket.h:91-93) bound what a client can send, so a malformed or hostile frame cannot allocate unbounded memory.

Resize must ride the data stream (ordering rule, cannot bend) #

Reflow on resize is a computation the client VT performs. It is not bytes in the stream. A client that learns sizes out of band while replaying bytes reaches a different grid than the original. So RESIZE frames sit in the data stream, in order, interleaved with BYTES. A replaying client reconstructs correct state, then applies its own final resize and reflows locally.

Two distinct directions:

  • Client → daemon on control: make the pty this size. Controlling client only.
  • Daemon → client on the data stream: the pty became this size at this point in history. Every client receives this.

EXIT also belongs on the data stream, since it is the ordered end of that pane's history. Control separately notifies the mirror so structure updates.

The log is the source of truth, sockets are hints #

The daemon reads the pty, appends to the in-memory log, then tries to write to each attached client. The daemon never blocks the pty reader thread on a slow client. Backpressure uses the verified QWebSocket API: check bytesToWrite() (qwebsocket.h:89) before writing, and when a client exceeds its cap call pauseMode/resume (qwebsocket.h:64,63) or close it outright with CloseCodeTooMuchData (1009) and let it re-attach from its last seq. Lossless, because the log still holds everything.

Backpressure comes free: an invisible pane is not attached, so it consumes no socket bandwidth. A visible pane that falls behind resyncs by seeking.

The pty reader thread (16 KB chunks, Pty.cpp) and the daemon's WebSocket writer must not contend. Keep the log append and the fan-out on the daemon's event loop, fed by the reader thread's queued signal, exactly as readyRead feeds vt_write today (TerminalQuickItem.cpp:548-571).

Reattach is a seek. attach(paneId, fromSeq=N) returns exactly the missed records. Cold start replays from the log head. Replay is progressive: the UI's VT is renderable at any point mid-stream, so the client paints as bytes arrive and never waits for a state dump.

Below-floor attach. When fromSeq is below the trimmed log floor, the daemon replies with the real floor and the client accepts partial history with older scrollback lost. This is where the agreed trim policy lands.

UI-side change is small #

TerminalQuickItem holds one Pty* m_pty (TerminalQuickItem.h:313) and connects exactly two signals (TerminalQuickItem.cpp:80-81): readyRead(QByteArray) and processExited(int).

Settled (user, 2026-09-08): the UI uses a PtyClient class with the same shape as Pty — the same nine methods and two signals (Pty.h:29-136) — that transparently talks to the daemon's Pty over the wire protocol. The goal is to minimize client changes.

No shared abstract base class is extracted. Pty and PtyClient never coexist in one process: the daemon compiles Pty, the UI compiles PtyClient, and nothing must hold a pointer to both. So no virtual dispatch on write() (called on every keystroke), and Pty stays final (Pty.h:29). PtyClient matches Pty's surface by convention, not inheritance.

An earlier draft of this note proposed extracting a 9-method interface. That is superseded by the PtyClient approach.

The 160 KB of renderer code does not change. The full PtyClient design — including the close()-means-detach requirement, the paneId injection gap, the synchronous-accessor problem, and lazy auth — is normative in notes/daemon-contract.md under "PtyClient: the UI-side pty".

AppModel relocation #

QML cannot bind to QObjects in another process. So the daemon holds the authoritative structure, and the UI keeps its existing QObject tree as a wire-driven mirror.

Settled requirement (user, 2026-09-08): the daemon's classes must mirror the current QObjects 1:1 in both data and behavior. The daemon's Workspace / XenoTab / SplitNode / SplitContainerNode / PaneNode / AppModel carry the same fields, the same tagged-sum kind() + exhaustive visit(), the same NodeLocation path index, the same ActivePath shape, and the same mutation methods with the same guards (insertChild, takeChild, setRatios, movePaneBeside, swapPanes, movePaneToTab, movePaneToNewTab). The difference is only that the daemon's copies are plain structs/classes rather than QObjects, so they carry no signals and no parent-child ownership.

Why 1:1 matters:

  • All AppModel manipulation logic moves to the daemon by changing the storage substrate, not by rewriting the algorithms. The guards and the structural rules are copied, not reinvented.
  • The UI mirror and the daemon core stay diffable against each other. A divergence bug is visible by comparing two nearly identical classes instead of two different designs.
  • tst_appmodel.cpp (43.6 KB) and tst_splittree.cpp retarget at the daemon core with minimal edits, because the API under test is the same API they test today.

What moves: mutations, guards, the path index, id generation, session serialization. What stays in the UI: the QObject wrappers QML binds to, PanePool, controllers, XenoSettings.

Rejected alternative: daemon owns QObjects and the UI builds a generic proxy for QML to bind through. That would require rewriting every QML binding over containerNode.children/.ratios/.axis, modelData.isContainer, Workspace.kindWord. Far larger, no benefit.

Single writer (agreed). The daemon is authoritative. Structure mutations round-trip and the mirror applies on notification. No optimistic local edits. A local socket round trip is microseconds, structure changes are rare next to byte traffic, and divergence between two clients would be far worse.

Audit needed: any QML or controller code that mutates structure and reads the tree back in the same frame. Mission-control drag and split operations are the likely candidates.

What moves where #

Item Owner after split
Pty, master fds, shell children, pids daemon
per-pane byte log + seq numbering daemon (memory only)
workspaces → tabs → split tree → pane ids daemon
PaneRecord spawn fields (spawnCwd, spawnCommand, spawnInitialCommands, spawnTitle, restart, restartArgv) daemon
pane cwd (foregroundCwd via lsof, Pty.h:74-79) daemon
SessionStore / session.json daemon
GhosttyTerminal, GhosttyRenderState UI, per client
renderer, glyph atlas, shaders, selection, kitty UI, per client
PanePool (live TerminalQuickItem re-parenting) UI
XenoSettings (font, sidebar) UI
XenoSettings.processRestartMatches UI, but pushed to daemon, since the daemon now owns foreground pids at quit time
Commands registry + controllers UI (Quick-dependent)
ActivePath (focus/selection) UI, per client. Each client has its own focus

ActivePath stays per-client deliberately. Two clients viewing the same session should not fight over which pane has focus.

Platform #

POSIX only: macOS + Linux. Windows support is revoked (decision 8).

The ConPTY branch in Pty.cpp/Pty.h (#ifdef _WIN32) is dead code that the build never compiled, since flake.nix:10 lists only x86_64-linux, aarch64-linux, aarch64-darwin. Leave it in place during the split; do not touch it. Removing it is a separate cleanup decision, not part of this work, and the daemon inherits the POSIX path only.

Daemon needs QtCore + QtNetwork + QtWebSockets, runs headless on macOS and Linux.

Nix: add a xenomorphicd derivation beside app.nix. Omit wrapQtAppsHook (no Qt plugins to resolve) and omit the ${ghostty}/lib rpath (daemon does not link ghostty). Add qt6.qtwebsockets to both the daemon and the app buildInputs.

Auth is the local per-boot token file. The daemon binds 127.0.0.1 and never 0.0.0.0. Remote auth is deferred (see notes/daemon-remote-tailscale.md).

Phases #

  1. Byte broker. Move Pty into a new xenomorphicd binary with the in-memory seq log + QWebSocketServer (local, token auth, 127.0.0.1). Add support/DaemonLink (owns /ctl, lazy auth, cached in AppModel) and terminal/PtyClient (same shape as Pty, owns its /pane/<id> data connection). App auto-spawns the detached daemon on launch and reconnects to a running one. Delivers: shells survive closing the app.
  2. Structure in the daemon. Move AppModel logic to a shared plain-data core mirroring the QObjects 1:1. UI AppModel becomes a wire-driven mirror. SessionStore and structure notifications cross the wire. Delivers: any client sees the real layout.
  3. Remote (deferred). Not scheduled. Design and pickup checklist are in notes/daemon-remote-tailscale.md. Because phase 1 standardized on WebSockets, this is additive: a second listener binding plus a Tailscale whois authorization gate on accept, no new transport or codec.

Sequencing note: phase 1 alone delivers the main ask and is small. Phase 2 is the large refactor. Doing phase 2 first would block the simple win behind it. Phase 3 is explicitly out of scope for now.

Known risks #

Extra hop on the hot path. Today readyRead is a queued cross-thread signal into vt_write (TerminalQuickItem.cpp:548-571). Adding a socket hop costs tens of microseconds per 16 KB chunk. lastChunkReadUs (Pty.h:85) already instruments this, so measure the regression.

Trim boundary must be parser-safe (concrete mechanism). Trimming at an arbitrary byte makes a cold attach below floor start mid escape sequence, which corrupts the client's VT. The daemon has no VT, so it cannot ask one whether it is at ground. Resolution: the daemon tracks a tiny byte-scanner state, not a terminal. It only needs "am I inside a CSI / OSC / APC / DCS sequence, and am I inside a UTF-8 multibyte char". Roughly fifty lines. It trims only at points where the scanner reports ground, and only at record boundaries so a BYTES payload is never split. ghostty's standalone osc.h is precedent for shipping a sub-parser without the terminal.

Cap the log at the same 10 MB the client VT already allows (TerminalQuickItem.cpp:287), since the log holds the same bytes the scrollback holds, not extra memory. On overflow a cold attach gets the recent tail only, and the daemon replies with the real floor.

Two clients on one pane at different sizes (deferred). One pty has one TIOCSWINSZ. Only becomes reachable once a second client can attach, which is the deferred remote work. Working rule: the controlling client sets the size, others render the true pty grid and scroll or scale. Decide when remote lands; see notes/daemon-remote-tailscale.md.

Titles cross the boundary. Titles arrive from the UI's VT via TITLE_CHANGED (TerminalQuickItem.cpp:320-321), and that stays true after the split: each client runs its own VT, so it derives its own title. No daemon work needed. Only the deferred remote client, which may want titles for panes it has not attached to, would need a small OSC sniffer in the daemon. ghostty ships a standalone osc.h for exactly that.

Renderer must not move. TerminalSceneRenderer reads GhosttyRenderState by direct pointer on the render thread under m_stateMutex (TerminalQuickItem.cpp:756-783, 2661). render.h documents this as intentional shared-memory access for a renderer thread sharing a lock with an IO thread. There is no serialization seam. This rules out "daemon owns the VT and ships frames" permanently.