anomalybait.me / subtokenling
a tiny pet with an even tinier brain, living in your browser
Star
Fork 0
Watch
subtokenling docs extending.md
11 kB
Markdown
at main

Extending subtokenling #

Anything the model must say or understand is a retraining event: the weights only know what was in the data. Anything purely visual or in the simulation is not. Each recipe says which it is.

Rules shared by the browser and the host live once in core/ (see its guide); never add them to web/ or server/ code. After any change that touches world.py or core/, run npm test, build tokenling-contract and run node server/tests/parity.mjs.

Retrain loop #

  1. Update train/world.py and core/src/world.rs together. Run node test/parity.mjs, build tokenling-contract and run node server/tests/parity.mjs.
  2. Update the teacher prompt in train/gen_data.py (it builds word lists from world.py, but scenarios and examples are written out) and the validator so it accepts exactly the new grammar.
  3. Probe: gen_data.py probe.jsonl 6 2, read the rows, fix prompt or checks.
  4. Generate the full set (free the GPUs first, see deployment.md), train, run eval_pet.py, export, publish as a new model id, update site/release.json and deploy.

Add an emote #

Model-facing:

  • EMOTES in train/world.py and core/src/world.rs.

App-facing:

  • FACE in web/scene.ts (eyes, mouth) and any motion branch in drawPet;
  • an entry in ACT_WORD in web/app/text.ts and, if it has a floating icon, the emote branch of playEffect() in web/app/animation.ts.

Retrain so the model produces it. A new word the model has not seen will not appear. (For a quick visual-only animation, add it to FACE/drawPet and trigger it from a care action with animate(); no retraining.)

Add a tool #

An action tool (like [gift] or [walk:left]): add the name to EFFECT_TOOLS in both world files (and a value list such as WALKS, which test/parity.mjs compares), validate its arguments in check_chat(), apply it in Engine::apply_line in core/src/engine.rs (emit an effect for the LCD and play it in playEffect()), show a chip or an action word in chipsFor() / ACT_WORD, describe it in the teacher prompt, then retrain. Prefer a tag for anything the pet decides; the app should only carry out the decision.

A new offer (something the owner proposes and the pet accepts or refuses): add the event line to event_patterns() and offer_patterns() in world.py, the builder to events in core/src/world.rs (and event() in core/wasm/src/lib.rs for the parity test), branches in the Care command and answer_offer in core/src/engine.rs, a NO_POOL entry and scenarios in gen_data.py, and a case in the parity test's event list. Then retrain.

A lookup tool (like [clock]): in addition to the above, add it to LOOKUP_TOOLS and tool_result() with identical logic in world.py and core/src/world.rs, and update the two stop regexes: LOOKUP_END in core/src/world.rs (both brains stop with it) and in train/eval_pet.py. Mention it in the static part of the teacher prompt and add scenarios that call it. Results must be deterministic functions of the prompt state, because the validator replaces whatever the teacher wrote.

Add a dream #

Add the value to DREAMS in both world files and provide its sprite. Foods and gifts are included automatically. dream_thing/dreamThing supplies the day-log and chip label. Update sleep teacher coverage and validation, retrain, and ship a new model. Preserve the closed sleeping grammar and bounded dream log. Body-only wake tuning uses wake_chance in core/src/world.rs and world.py; parity covers it.

Add or change a food, gift or want #

  • Lists: FOODS, GIFTS or WANTS in both world files.
  • Art: add a sprite to SPRITES in web/sprites.ts (rows of 8 characters, see KEY for the palette) and, for wants, WANT_ICON in web/app/config.ts.
  • Behavior: feeding effects are in answer_offer in core/src/engine.rs.
  • Foods and gifts appear in prompts, so this is a retraining event. A food that exists in the app but not in the training data confuses the model.

Add a life stage or change timing #

  • Names: STAGES in both world files; voice text in STAGE_STYLE in gen_data.py; sampling weights in rand_state().
  • App: STAGE_AGES in core/src/body.rs, a SIZE entry and decorations in scene.ts.
  • The prompt only contains the stage name and day number, so changing ages alone is not a retraining event, but new stage names are.

Change the simulation #

Rates, poop timing and sleep rules are constants in core/src/body.rs (simulate_body); care, offers, scheduling and the day log are in core/src/engine.rs. Minigame tuning is in web/app/game.ts (stepGame()). Keep them in sync with pet-simulation.md; node server/tests/parity.mjs checks that both builds still agree. If you change the stat words or thresholds (levelWords), it is a retraining event: the model learned those exact words.

Change the look of the page #

The host page is site/index.html (styles inline, tokens on :root, light and dark). Interactive markup and event handlers are Preact components in web/components/; body and model behavior stay in web/app/. Pixel art lives inside the LCD canvas and the sprite icons. Keep focus rings, the 360 px layout, reduced motion and the no-JavaScript notes working, and run node test/contract.mjs if you rename or remove an element.

Add a stored field #

Add it to PetState in core/src/types.rs, fresh() in core/src/save.rs and PetState in web/types.ts. A new top-level key is filled for older saves from a new save's defaults, and a new setting whose absence means "off" needs nothing. Nested changes or renames need VERSION bumped and a step appended to MIGRATIONS in core/src/save.rs (it receives the previous version's object and must return one with v one higher). Never make load() accept a newer version: it shows the bad egg instead, and the save stays untouched. Test with an old save (export one before the change) and document it in pet-simulation.md.

Ship a feature only newer brains have #

When a feature needs training (a new tool, offer or event line the old brains never saw), the app must not use it with an older brain the visitor kept. Name the feature, list it under features in site/release.json (and in each older entry that has it), describe it in FEATURE_TEXT in web/app/config.ts for the upgrade dialog, and gate it: brainHas("name") for component disabled/class/title props, data-needs="name" for inspection, ACT_NEEDS for care actions, and brainHas("name") around any other code path that writes the new lines. A change to the prompt format itself cannot be gated this way: empty older instead (deployment.md).

Change the base model #

The small-model matrix in train/experiments.json now has runnable native GPT-2/Pythia, vocabulary-reduced Pythia and tied standard Llama experiments; this does not select or publish a replacement. The inherited baseline v1 had 355M parameters; v2–v6 have 124M, and the current published file is 97,276,118 bytes. Those are baseline facts, not target sizes or architecture requirements. The current engine remains WebAssembly.

The source still has baseline-architecture assumptions. A new checkpoint path or an automatic tokenizer/browser loader does not make the pipeline universal. For each candidate, adapt the affected contracts rather than papering over unsupported shapes:

  • train/prepare_model.py, train/model_data.py and train/train.py: generic causal-model loading does not imply support for every tokenizer. Current preparation supports real ByteLevel BPE native/prefix controls and scratch Llama. Preserve strict tokenizer character offsets/loss boundaries, real end/pad tokens, declared context, unchanged canonical spans and reproducible checkpoint/config receipts. Explicit text-only pretraining must never become a bypass for masked pet data.
  • train/eval_pet.py: model loader, context trimming, special/end tokens, decoding and past-cache API. Preserve the app's offer/sleep/idle/diary and deterministic lookup loop, including Name: without a trailing space.
  • train/export.sh and train/quantize.py: verify exporter support, real constant/Gemm/MatMul topology, configured tied versus untied tables, quantization axes/block format and last-token slicing. The current quantizer derives these from supported graph/config contracts, not an initializer alias or the baseline's 48 weights / [768, 50257] head. Unsupported or ambiguous shapes fail; add a real adaptation, not a name shim.
  • train/onnx_utils.py, train/check_onnx.py and train/quant_eval.py: supported separate rank-four GPT-2/NeoX/Llama past tensors derive KV heads, head width, dtypes and matching presents from actual metadata and config, including GQA. Merged/unfamiliar caches still require adaptation. Preserve fp32 numeric/greedy parity, cached-versus-fresh parity for every dtype, and full-sequence masked loss. Quantized greedy drift is a measurement, not an automatic requirement to match every fp32 token.
  • web/brain.ts, web/app/brain.ts and the generation calls in web/app/conversation.ts/web/app/diary.ts: supported browser model/config, typed worker messages in web/types.ts, the vendored API boundary in web/runtime-types.ts, context budgets (currently 1024 react / 512 idle), tokenizer and merged-boundary handling such as ][, stop/end-token rules, q4 filename and session options. Verify the resulting artifact with the pinned runtime; do not invent a fallback engine or silently replace pet decisions. The worker source is TypeScript but its emitted URL stays brain.js.
  • server/src/brain.rs: native support is explicit for GPT-2, NeoX and Llama/GQA separate rank-four caches. Configure the export dtype, preserve actual vocabulary/EOS/context and metadata validation, and exercise real native generation plus private HTTP/transfers before hosting another model. Keep model-ID pinning; a successful native warmup is not a quality gate.
  • train/world.py, core/src/world.rs, train/gen_data.py and the data: retain the exact frozen wire markers and closed grammar for compatible weights. If those contracts change, update both world files, teacher/validator and data, retrain, and empty the release's incompatible older list.

Before publishing, require prompt parity and data-rule checks, behavioral evals and transcript review (offers, sleep/dreams, recall, idle voice and diaries), quantization quality versus the candidate's unquantized checkpoint, measured download size, load/prefill/decode speed and page memory. Run bench.html and the full real-browser flow in testing.md: fresh download/hatch, cached and offline reload, every lookup, diary, upgrade/incompatible-brain handling, legacy-save preservation, alerts/install and the 360 px surface. Identify the hardware, threads, dates and unmeasured devices; baseline receipts are not proof of a candidate's performance. Publish only a proved new immutable MODEL_ID, update release size/runtime/features and visitor-facing details, and keep the historical receipts labelled as inherited baselines.