This optional host runs a pet while its browser is closed. Axum provides the private HTTP API, SQLite keeps the complete save, and native ONNX Runtime runs the existing exported brain. The pet's rules come from the shared crate in core/, the same code the browser runs as WebAssembly in local mode. The browser renders the room and catch game. Local mode remains a standalone offline app. Owners choose local or remote at setup and can transfer either way in settings.

Remote mode sends the save, including chat, memories and diary, to the chosen host. Choose a trusted host. Each pet has a random 256-bit access key kept in its browser; the server stores only a SHA-256 hash. Every read, command, transfer and deletion requires that key. There is no public pet listing. Saves, prompts, generated text and keys are not logged. ATProto sign-in and multiplayer are later work.

Build and configuration #

Use Rust 1.88+ and the Cargo lockfile. Supply a compatible native ONNX Runtime library separately: the binding requests API 27 (1.28+). The library must support the published graph's blocked DequantizeLinear at opset 21. QDQ fusion and prepacking are disabled, matching the browser and native probe. A real one-token warmup must pass before HTTP starts. Models are not downloaded.

cargo build --release --locked --manifest-path server/Cargo.toml --bin tokenling-server
export SUBTOKENLING_LISTEN=127.0.0.1:$PORT
export SUBTOKENLING_DB=$STATE/pets.sqlite
export SUBTOKENLING_MODEL_DIR=$DATA/release/MODEL_ID
export SUBTOKENLING_MODEL_ID=MODEL_ID
export SUBTOKENLING_ALLOWED_ORIGIN=https://APP_HOST
export ORT_DYLIB_PATH=$ORT_LIBRARY
server/target/release/tokenling-server

Variable	Meaning
`SUBTOKENLING_LISTEN`	Required socket address, including port. Use a private listener behind HTTPS for other devices.
`SUBTOKENLING_DB`	Required SQLite path outside Git; contains private live state.
`SUBTOKENLING_MODEL_DIR`	Required export directory with `tokenizer.json`, `config.json`, and the selected ONNX graph plus any referenced external tensor files.
`SUBTOKENLING_MODEL_DTYPE`	`q4` (default), `q8` or `fp32`: selects exactly `onnx/model_q4.onnx`, `onnx/model_quantized.onnx` or `onnx/model.onnx`. Missing files fail startup; there is no precision fallback.
`SUBTOKENLING_MODEL_ID`	Required release ID, matching the browser's selected brain. Transfers never silently upgrade it, and the host refuses to start while a hosted pet selects another brain.
`SUBTOKENLING_ALLOWED_ORIGIN`	Required comma-separated browser origins, with scheme/port. Wildcards are rejected.
`ORT_DYLIB_PATH`	Required native ONNX Runtime library path.
`SUBTOKENLING_THREADS`	Positive CPU intra-op thread count; default 4. Inter-op stays at one.
`SUBTOKENLING_GPU_DEVICE`	Optional logical CUDA index after `CUDA_VISIBLE_DEVICES`, requiring a CUDA-enabled ORT library and an admitted GPU allocation. Default CPU only; GPU initialization failure leaves CPU available.
`SUBTOKENLING_MAX_PETS`	Registered-pet bound, including parked saves; default 64. This is not measured serving capacity.
`SUBTOKENLING_WEATHER_SEED`	Deterministic weather seed; default empty.
`SUBTOKENLING_FEATURES`	JSON list of trained release features; default `[]`.

The native loader explicitly supports model_type values gpt2, gpt_neox and llama; accepting an export is not a behavioral approval of a replacement brain. Configured layer/head/hidden dimensions, Llama head_dim and KV heads (including GQA), vocabulary and scalar/list eos_token_id are validated. A positive max_position_embeddings or n_positions is required; if both are present they must agree. Context fitting uses the smallest of the request, model positional capacity and the shared app limit of 1024 tokens, reserving the output budget. Old history lines are dropped; an unfit header/prefix or final tokenized prompt fails rather than overrunning the positional model. Conflicting dimension aliases, unsupported architectures and out-of-vocabulary EOS/tokenizer IDs fail. Padded logits vocabularies may exceed tokenizer size.

Use an unmerged with-past graph: input_ids is required; attention_mask, position_ids, token_type_ids and rank-one/two cache_position are optional int64/int32 inputs. Logits must be float32, rank three [1, sequence, vocab]; all-token and last-token-only outputs are supported. Actual runtime vocabulary must equal config, including when the metadata vocabulary is dynamic. Every layer must expose separate rank-four past_key_values.N.key/value inputs and uniquely matched present.N.key/value or present_key_values.N.key/value outputs, with unchanged float16/float32/float64 cache type. Cache layout must be [1, KV heads, sequence, head width], with fixed head axes and a dynamic sequence axis; ambiguous dynamic head axes, fused/transposed caches, missing pairs and extra inputs/outputs are rejected. Runtime metadata checks exact cache growth on every step without copying CUDA caches to the CPU. Weight precision does not determine cache precision. CUDA outputs remain device-resident; only logits are bound to CPU for sampling.

All host-specific environment variables use SUBTOKENLING_*; old TOKENLING_* settings are not aliases. The model ID remains pinned independently of dtype. No download, network inference or substitute brain is attempted.

Unix database/WAL/SHM permissions are restricted to 0600. Protect the directory and backups too. Run one host process per database. TLS belongs at the reverse proxy; API responses use Cache-Control: no-store and the configured CORS allow-list. Loopback HTTP is supported for previews. Other browser endpoints must use HTTPS.

A host runs one model for every pet. To change SUBTOKENLING_MODEL_ID, have owners bring hosted pets local first, or run the new model as a separate host with its own database; startup reports how many pets block the change and modifies nothing.

As a long-running service, run it as a dedicated unprivileged user. Keep the model and native runtime read-only, make only the database directory writable, listen on loopback, and terminate HTTPS at a reverse proxy. A real one-token warmup runs before the listener opens, so a health check also confirms the brain loaded. The browser's recovery checkpoint is the owner's copy; back up the database separately if server-only progress must survive disk loss.

REMOTE_ENDPOINT in site/build.sh optionally supplies a default host URL. Without it, owners enter their server address. It does not change the default local execution mode.

API and ownership #

GET /api/health advertises protocol 1, release/features and available workers. Every other route requires Authorization: Bearer KEY:

Route	Behavior
`POST /api/pets`	Prepare an inactive save with a random UUID/key. Retry never overwrites an existing pet.
`GET /api/pets/ID`	Private save, epoch/revision, effects, game and thinking status.
`POST /api/pets/ID/commands`	Owner command, expected epoch and unique operation ID.
`POST /api/pets/ID/transfer`	Activate an uploaded local save remotely, or park the executor and return its latest save.
`DELETE /api/pets/ID`	Delete the pet and operation receipts. Late replies cannot recreate it.

Save v2 adds hosting (mode, brainId, timeZone). Real v1 saves migrate to local mode without changing pet data. The live key remains subtokenling.v1; the compatibility Web Lock stays tokenling.v1, and legacy save/archive bytes retain the bridge described in pet-simulation.md. Connection keys, identifiers and transfer intent are separate in subtokenling.remote.v1, and exported local recovery saves omit them. Pending recovery stops live separately in subtokenling.remote-stops.v1 so switching hosts cannot discard an unreachable old host's key. They retry every 30 seconds while the owning tab is open and on reload/reconnect.

Local-to-remote prepares an inactive record, then persists remote intent before activation. A reload reconciles a lost activation response. Remote-to-local atomically parks the executor, advances its epoch and returns the latest save. Parking also fences a prepared record, so an old activation cannot revive it. The browser resets its body clock before resuming so remote time is not replayed. A destination-device ID prevents another connected browser from automatically starting a local copy after that handoff.

The host owns all state in remote mode. Each pet has one generation at a time. Committing requires the same epoch and relevant context revision; new owner interactions cancel stale work. Operation IDs deduplicate retries. Browser sync ignores older snapshots, and local replies check their execution revision before applying effects after a mode switch.

An unreachable host permits explicit recovery from the last browser checkpoint. Newer server-only progress may be missing. Its copy can keep thinking until the pending stop reaches it; its eventual snapshot never overwrites newer local progress. Graceful transfer fences ownership. Offline recovery cannot prove an unreachable executor has stopped.

Simulation and workers #

The host links the shared engine for body rates, growth, care/offer effects, the wake formula, closed tags, deterministic lookups, turn scheduling, the diary and trained prompts; it adds ownership, epochs, error back-off and workers around it. Stored rows keep the engine's fields at the top level. The model chooses pet actions; there are no canned replies or random wandering. The browser plays visual effects without applying care effects again. Catch starts only after acceptance; its result commits once with the game ID. An abandoned game expires without inventing a score.

CPU is resident. Before timing samples exist, an optional CUDA session prefers reactive turns while CPU prefers background work. Once both workers have estimates for that request shape, observed or derived as described below, the host compares predicted queue completion times, including work already running. An idle CPU can wait when the GPU is expected to finish a small queue sooner. Larger queues can still use both lanes. Sessions run serially per worker. CUDA keeps KV tensors on the GPU through I/O binding; only logits return to the CPU for sampling. Each generation starts with an empty cache, and cancellation releases that request's bindings. TF32 is disabled so the GPU uses FP32 math for the expanded weights. Initialization failure leaves CPU serving available; a later GPU failure disables that worker and requeues its turn on CPU.

Timing estimates are nine numeric cells per worker: reactive/idle/diary crossed with three approximate prompt-size bands. Only successful, uncancelled whole turns update an estimate: a plain mean of the first five samples, then an exponentially weighted mean (20% new sample), so one unusual first turn does not dominate. Prompt size uses bounded UTF-8 byte counts divided by four, not exact tokenization. When only one worker has run a shape, the other worker's estimate is that cost times the least favourable speed ratio between the workers on shapes both have run. An unseen GPU shape can then wait for a clearly faster GPU instead of repeatedly spilling to an idle CPU. Without such a shared shape, unseen work keeps its original affinity, and a busy shape with no estimate is treated as having unknown completion time. No timings, record IDs or text are persisted by the router. Estimates reset on process restart.

The planner applies a 10% CPU improvement margin to limit timing noise. Queued background work moves ahead of new reactive work after 30 seconds; running generation is not preempted for this promotion. The wait survives context edits and resets after a mode/epoch change or work-kind change. These constants are generic implementation choices by reasoning, rather than measured service latency guarantees. Batching and production sustained capacity are not established. This native ONNX path is distinct from the earlier PyTorch probe. The opt-in checks below measure its real private HTTP adapter and both workers.

Foreground sync polls once per second, less often while hidden. The browser keeps a recovery checkpoint and does not run local simulation in remote mode. Existing alerts remain browser/device managed; there is no push service or guaranteed closed-browser notification delivery.

Checks and source map #

cargo fmt --check --manifest-path server/Cargo.toml
cargo test --locked --manifest-path server/Cargo.toml
cargo test --locked --manifest-path core/Cargo.toml
cargo build --locked --manifest-path server/Cargo.toml --bin tokenling-contract
core/wasm/build.sh
node server/tests/parity.mjs

Scripted generators exist only in tests. Tests cover authentication, background turns without browser polling, both transfers, old-reply fencing, deduplicated care, prepared-record fencing, memory eviction, GPU failure fallback, restart persistence, lane preference/serialization, body rules and sleeping offers. Native/Python parity covers 300 states and 15 lists, lookups and wake probabilities; the replay then runs one 31-step engine scenario natively and in WebAssembly, in two time zones, and requires identical engine JSON after every step. Browser checks are in testing.

For a direct CPU export smoke (including prefill and cache reuse), unset SUBTOKENLING_GPU_DEVICE, build the probe and send a generation object. Choose the matching export directory and precision explicitly; repeat for each export. The probe performs the same startup warmup as the host. Its diagnostic stdout contains the prompt: never use a private save as a public benchmark fixture.

cargo build --release --locked --manifest-path server/Cargo.toml --bin brain-probe
export ORT_DYLIB_PATH=$ORT_LIBRARY SUBTOKENLING_MODEL_DIR=$MODEL_EXPORT
export SUBTOKENLING_MODEL_DTYPE=q8 SUBTOKENLING_THREADS=4
printf '%s\n' '{"header":"","lines":[],"prefix":"hello little friend","mode":"line","maxNew":4,"temperature":0,"topK":40,"topP":0.92,"ctx":1024}' | server/target/release/brain-probe

For a fixed-output compute diagnostic, append --benchmark-tokens 32 to brain-probe. It greedily generates exactly that many tokens, ignoring EOS/newline/lookup stops only for this diagnostic. Normal host/probe generation is unchanged. Compare prompt token counts, prefill milliseconds and decode tokens/second separately; forced output is not a useful pet response. The all-model benchmark records both this workload and normal stopped generations using generic fixtures.

For real-model HTTP checks, compile before borrowing a GPU and supply explicit runtime/model inputs. Ordinary tests intentionally skip this native test:

cargo test --release --locked --manifest-path server/Cargo.toml --test native --no-run
export ORT_DYLIB_PATH=$ORT_LIBRARY SUBTOKENLING_MODEL_DIR=$MODEL_EXPORT SUBTOKENLING_MODEL_ID=$MODEL_ID
export SUBTOKENLING_MODEL_DTYPE=q4 SUBTOKENLING_THREADS=8 SUBTOKENLING_NATIVE_PROFILE=mixed SUBTOKENLING_GPU_DEVICE=0
export SUBTOKENLING_NATIVE_OUT=$RECEIPTS/new-mixed.json
cargo test --release --locked --manifest-path server/Cargo.toml --test native -- --ignored --test-threads=1

SUBTOKENLING_NATIVE_PROFILE is required: cpu, gpu or mixed. In the GPU-only test profile the CPU worker is present but disabled; production keeps CPU available. SUBTOKENLING_NATIVE_COUNTS defaults to 1,4,8,16, each a simultaneous burst whose requests queue behind available workers. SUBTOKENLING_NATIVE_ROUNDS defaults to 3. Short and varied prompt-length bursts are tested separately. An explicit greedy wrapper makes profile comparisons repeatable; production sampling remains at 0.85. Three prompt cases include a near-limit context, and each has five measured repetitions after a warmup. Checks include real clock continuation, cancellation during inference and complete-state parking. Receipts identify model dtype and hash the selected graph, and contain hashes/timings, never keys/saves.

SUBTOKENLING_NATIVE_STEADY_SECONDS optionally runs eight sequential owner streams after the controlled bursts (default 0, maximum 300 seconds). SUBTOKENLING_NATIVE_STEADY_TEMPERATURE defaults to 0.85 for this phase. It records per-pet progress, latency and generation counts and finishes current requests before parking. Chatter is disabled for the controlled comparisons; ordinary body/diary behavior still applies during the steady phase. Partial numeric progress/failure receipts are written beside the requested output.

The separate startup check requires a CPU-only library and verifies actual server fallback, private access, CORS/no-store, a real CPU reply, transfer and deletion. Required flags are --server, --library, --model, --model-id, --out; optional --threads=4 and --dtype=q4|q8|fp32 (default q4). Its fresh output directory contains a disposable private database and must stay outside Git. Both tests bind ephemeral loopback listeners.

Path	Role
`src/main.rs`	Configuration, native workers, HTTP/CORS and startup
`src/host.rs`	Private API, dispatch of the engine's tasks to workers, and commit fencing
`src/routing.rs`	Bounded numeric timing estimates, queue plans and wait aging
`src/store.rs`	SQLite state, key hashes, epochs and idempotent operations
`src/simulation.rs`	`HostedPet`: the shared engine plus ownership, epoch and error fields
`src/brain.rs`	Tokenizer, native inference and sampling, using the shared fit and stop rules
`src/bin/contract.rs`, `tests/parity.mjs`	Prompt/list/lookup/wake parity with Python, and the native/WebAssembly engine replay
`src/bin/brain-probe.rs`	Generation object or array from stdin, reusing one session; required `ORT_DYLIB_PATH`, `SUBTOKENLING_MODEL_DIR`, optional `SUBTOKENLING_MODEL_DTYPE=q4`, `SUBTOKENLING_THREADS=4`, `SUBTOKENLING_GPU_DEVICE`. Reports token counts/timing; diagnostic stdout includes the prompt.
`tests/host.rs`	API, state-ownership, body and generation regressions
`tests/native.rs`	Opt-in real CPU/GPU comparison and concurrent private HTTP/ownership checks

The prototype has no ATProto accounts, multiplayer, host-to-host visits, public-host abuse controls or full app/model export package. Native CPU and browser/native checks are recorded in the test receipts. Public deployment, sustained capacity and model batching remain unqualified. The Axum API and ort sessions describe dependency interfaces; Cargo.lock pins the build. See ORT CUDA requirements for matching runtime/CUDA/cuDNN versions, and I/O binding for the device-resident output contract.