# Data, training and evaluation Scripts: `train/gen_data.py`, `train/mix_data.py`, `train/train.py`, `train/eval_pet.py`, `train/quant_eval.py`, `train/teachers.sh`, `train/migrate_v6.py`, `train/prepare_model.py`, `train/model_data.py` and `train/experiments.py` with its pinned matrix `train/experiments.json`. Datasets, checkpoints, exports and logs live outside the repository (`$DATA`). The small-model experiments use PyTorch 2.14.1, transformers 4.57.6 and tokenizers 0.22.2. CPU and ROCm 7.2 preparation/training were exercised; CUDA uses the same Torch device API but was not exercised in this experiment. Use isolated environments (`$VENV` for training, `$EXP` for export); do not replace dependencies of an unrelated resident service. **Inherited baseline receipts:** the v1–v6 datasets, dates, hashes, losses, training times and browser observations below describe earlier baseline runs, not a newly trained subtokenling model. Baseline v1 had 355M parameters; v2–v6 have 124M. These counts and the 1024-token training/eval budget describe the current source path, not requirements for future brains. Base revision `607a30d` identifies the inherited v2–v6 base checkpoint in the receipts. Small-model experiments now prepare genuine GPT-2/Pythia warm starts and a standard tied Llama scratch candidate, with generic causal-model training and browser-aligned behavioral evaluation. This is not a published replacement or a universal-model promise. The initial screening did not have v6 weights or its training corpus: a newly fine-tuned GPT-2 control is an experimental reference, **not evidence of parity with inherited v6**. The later compact-model improvement receipt recovered and checksum-verified the published v6 q4 graph for independent inference comparisons, not its original Torch checkpoint or training data. See [the experiment recipes](#small-model-experiments) and [research note](small-model-research.md); publication still requires behavior, quality and real-browser size/speed/memory evidence. ## 1. Teacher data `gen_data.py OUT.jsonl N_CHATS N_DIARIES` asks a large model to write one chat log or diary entry per request and keeps only samples that pass validation. **State sampling.** `rand_state()` builds a random pet: stage (baby 2 : kid, teen, adult 3 each), two traits, random time, weather, stats, up to six memories drawn from sentence templates, a six-entry day log, shelf, toy, poop count, asleep and sick flags and a "yesterday" line. The header it renders is the exact production header (`world.header`). Since v2, "pooped" is kept in only 40% of day-log draws and the room has poop in 2 of 7 states (v1: 2 of 5), which reduces the poop skew described below. **Prompt.** A long static block states the rules (tag grammar, allowed values, no gender assumptions, one line per turn) and ends with a task block holding the names, the header, allowed event lines, length, personality, stage voice, the true lookup results and one to three scenario ideas from a list of 48 (greetings, asking the time, recall questions, gifts, comfort, boundaries, dancing, trivia, goodnight, offers, pats, wandering, idling and so on). Twelve of them exercise the pet's own decisions (`ACTION_SCENARIOS`), and 80% of chats include at least one. The rules explain offers (answer with exactly one `[yes]`/`[no]`, decided from the needs line and personality), pat reactions, walking and silent lines (at most two in a row), and ban stage directions. Since v3 they also state the memory rule: when the owner says something about themselves, and always on "remember that …", the pet saves it with `[remember:FACT]` in the same reply (even sleepy or just woken), while `[recall:…]` only looks memories up. `--focus memory` puts one of seven `MEMORY_SCENARIOS` into every chat; it was used for the v3 top-up. Since v4 the rules also say how to answer from memory: facts are saved with the words someone would ask about ("Sam is 29 years old"); the pet answers from the `remembers:` line when the fact is there, and otherwise looks it up with `[recall:WORD]` (one word that appears in the saved fact, because `recall` matches whole words) and answers from the result in the same line, or says it does not remember when the result is `{nothing}`. `--focus recall` builds every chat around one question about a fact from `RECALL_FACTS` (fifteen kinds: age, pets' and people's names, job, birthday and so on), placed older than the header's last three memories (55%, so only `[recall]` finds it), in the header (25%), or never told (20%). Other memories that share a word with the fact are left out so the lookup's result is the fact alone. `check_recall` rejects a row whose pet does not say the fact (or, for a fact never told, says it before being told); for older facts it also requires a `[recall]` that found something. Keeping the static block first lets the teacher's prefix cache serve it, which roughly doubled throughput. **v6 sleep and stage focus.** `--stage STAGE` forces the stage for both chats and diaries. `--focus sleep` samples mostly sleeping states with energy 0–95, plus awake-to-nap cases at energy 7–35. Talk, pat and lights-on outcomes use `world.wake_chance`, identical to the browser formula. The generator gives the teacher an ordered list of DREAM, AWAKE, NAP or sleeping OFFER turns and asks for a JSON array of pet lines. It assembles the app events itself and inserts `woke up` immediately after an accepted sleeping offer. The teacher still chooses each dream, mumble, awake reply and offer decision. Unfocused chats that start asleep also use this path. No dream text is canned. The teen voice is sassy but sincere, with varied wording instead of mandated catchphrases. Memory fillers avoid duplicating any starting-memory template, preventing conflicting ages in recall scenarios. Diary signatures use the actual pet name. Diary guidance explicitly identifies the pet as the subject of unlabelled log entries, keeps gift direction and owner speech intact, and does not turn remembered people or activities into events from today. Teen entries use 3–5 short sentences, with shorter logs kept brief. Dream entries are sampled in day logs; owner quotes are excluded from chat headers but kept in diary headers. **Validation (`check_chat`, `check_diary`).** A sample is rejected unless: - every line is an owner line, a pet line or an allowed event line; - owner and event lines contain no tags, braces or brackets; - owner lines do not narrate events: no event line in the owner's mouth (`Sam: * Sam patted Mochi *`) and no care narration (`Sam: fed Mochi a rice ball`, `gave you a bath`). Only the app writes events; - a `played catch with … (N stars)` result follows a `[yes]` to a catch offer; - every tag is in the grammar with an allowed value; `emote`, `want`, `gift` and `walk` values come from the fixed lists (a small alias table fixes common slips like `[emote:surprised]` → `wow`); - an offer event is answered by the pet's next line with exactly one `[yes]`/`[no]`, and `[yes]`/`[no]` appear nowhere else; - no more than two silent pet lines in a row, at most one `[walk:…]` per line, and no asterisks left after short `*stage directions*` are stripped; - lookup results are **replaced with the true values** from the state, and a `recall` the teacher claimed was empty or non-empty must agree with `world.recall`; `recall` searches the starting memories plus every `[remember:…]` earlier in the same chat, as the app does (through v2 it saw only the starting memories, so "remember X … recall X" taught `{nothing}`); - pet lines are under 220 characters with no stray brackets, and there are at least two of them; - a sleep state machine enforces one dream with at most four mumbled words; an offer refusal adds `[no]`, while a sleeping `[yes]` has no other tags and requires `woke up` immediately after. Exhausted sleeping pets accept food only when starving. Awake dreams, repeated sleep, wakes directly before offers or after a message/pat, and spontaneous instant wakes after naps are rejected; - memory tags name the owner; memory/note tags have no gendered pronouns; recall queries are at most two words and owner lines contain letters or digits; - narrow positive first-person hunger claims while full/stuffed are rejected; accepting medicine without sickness in the header is also rejected; fuzzier invented history and mood issues are handled by teacher guidance and transcript review, not claimed to be fully detectable by regex; - headers cannot contain conflicting remembered ages or chat `OWNER said:` entries, and chat weather must belong to the hour's day/night list; - diaries start `dear diary,`, use only valid `[doodle:…]` tags, and contain no remaining brackets or `doodle:` fragments or literal `- PET` signature. v1 rejected about 22% of samples, mostly for invented tag values and mismatched recalls. v2 rejected about 21%, mostly for `[yes]`/`[no]` outside an offer, then owner lines with tags and invented emote values. **Re-checking stored rows (`recheck`).** `gen_data.recheck(row)` applies the same line rules to a stored row, keeping the lookup results already in it. Fresh chat construction resolves lookups against effects committed by earlier pet lines: gifts append to the shelf without deduplication, saved facts become recallable after line completion, and cleanup clears poop. Effects earlier in the **same** generated line are still pending during its lookups, as in the real app. Stored-row rechecking preserves existing app-written results; when initial synthetic snapshots are available, replay them during corpus review and reject stale results rather than silently rewriting them. It also requires canonical text and a valid loss mask: span endpoints must be integers within the text, and their combined coverage must match the pet words/tags or diary body rebuilt by the parser. Equivalent merged, split, reordered or overlapping intervals are accepted; missing, shifted or incorrect coverage is rejected with `ValueError`, not silently repaired. Owner/event lines, headers, bare `Name:` prefixes and lookup results remain untrained; the model-generated separator after `Name:` is supervised. `train.py` re-checks every row before training and refuses to start if any row fails, listing the reasons; `--drop-stale` drops those rows instead. `mix_data.py` and `quant_eval.py` apply the same check, so data written for an older app cannot quietly teach behavior the app no longer has. When a rule or the app's behavior changes, add the rule to `check_body` and the next run reports every older row it catches. **v6 migration.** `python3 train/migrate_v6.py OLD.jsonl OUT.jsonl NEW.jsonl …` removes all old teen rows and filters visible chat-header quotes. It first rebuilds valid chat bodies without inventing events, preserving failed wake rolls and their dream replies. For legacy bodies needing a wake, it inserts a possible wake before a message/pat or after lights on only when an awake reply requires one; earlier no-wake dreams in mixed transcripts are retained. It also wakes after an asleep `[yes]`. Chat and diary text and loss spans are rebuilt with the current parser before re-checking, repairing their masks only in this explicit migration path and rejecting remaining stale rows. It does not fabricate replacement dreams. Old headers store only six log entries, so earlier filtered-out history cannot be recovered. Every new row is also checked; counts are printed by source, kind and first rejection reason. Optional `--exclude FILE` reads a JSON object mapping SHA-256 of row text to a confirmed manual-review reason; it matches before or after migration. The script uses arguments only, with no environment variables. `test/data_rules.py` checks it against a stand-in. History augmentation chooses donors whose final sleep state matches the next row's initial state, including both boundaries when it prepends two donors. Donor text remains untrained. This prevents stale donor naps from contradicting the trained awake continuation. **Spans.** Each row stores `text` plus `spans`, the character ranges that carry loss: the generated separator after bare `Name:`, the pet's words and tags, and the diary body after the fixed `dear diary,` prefix. Owner lines, event lines, headers, bare pet-name/colon prefixes and `{results}` are excluded. Older chat coverage excluding the separator requires the explicit [continuation-mask migration](#structured-curriculum-and-compact-distillation). ### Running the teacher `gen_data.py` needs any OpenAI-compatible chat endpoint (`--endpoints`, `--model`, or `TEACHER_ENDPOINTS` and `TEACHER_MODEL`; there are no defaults), but a shared model server is sized for a few concurrent users and is far too slow for bulk generation. For baseline v1 we ran dedicated batch servers, two independent vLLM instances of a W4A16-quantized 27B-class instruction model, one per GPU (24 concurrent sequences, 4,096-token context, prefix caching on), on ports 19400 and 19401. Each card holds the whole model plus a 60K-token KV pool, so the copies need no per-layer communication and fail independently. The generator sends each request to the less loaded instance. With about 50 workers the pair yielded 2.3 accepted samples per second (roughly 115–135 generated tokens/s per instance under load). Baseline v2 used the same setup: about 2.2 requests/s, 1.7 accepted chats/s with the stricter v2 validator. `train/teachers.sh start | wait | stop` is the generic version of that setup: one `vllm serve` per GPU with the same defaults, configured through environment variables (`TEACHER_MODEL` is required; `GPUS`, `PORT_BASE`, `MAX_SEQS`, `MAX_LEN`, `GPU_UTIL`, `VLLM`, `VLLM_ARGS` and `LOGS` are optional; the script header lists them). Put anything your model or vLLM install needs, such as quantization or attention-backend flags, in `VLLM_ARGS`. **Honest status:** the baseline v1 data was generated with a machine-specific launcher for the quantized 27B model. The generic script was tested end to end (start, health wait, a completion, stop, GPU memory released) with a small stand-in model, not re-run at full scale with the original teacher. Two instances were chosen over tensor parallelism by reasoning, **not by benchmark**. If your GPUs are linked by NVLink, tensor parallelism is cheaper than over PCIe and may be competitive. Measure `--tensor-parallel-size 2` with a larger sequence cap before assuming two instances are best for a bigger run. Sharing GPUs with other services is covered in `deployment.md`. ```sh train/teachers.sh start && train/teachers.sh wait python3 train/gen_data.py $DATA/v2/new.jsonl 9000 0 --workers 52 --seed 21 \ --endpoints http://127.0.0.1:19400,http://127.0.0.1:19401 --model TEACHER_NAME train/teachers.sh stop python3 train/mix_data.py $DATA/data.jsonl $DATA/v2/new.jsonl $DATA/v2/data.jsonl ``` For v2 the generator was stopped after 6,600 of the 9,000 requests (5,221 accepted chats), which was enough, to give the GPUs back sooner. Diaries were not regenerated: the diary format did not change. For the v3 memory top-up, with the same two-instance setup: ```sh python3 train/gen_data.py $DATA/v3/memory.jsonl 2200 0 --focus memory --workers 52 --seed 31 \ --endpoints http://127.0.0.1:19400,http://127.0.0.1:19401 --model TEACHER_NAME cat $DATA/v2/data.jsonl $DATA/v3/memory.jsonl > $DATA/v3/data.jsonl ``` All 2,200 requests ran in about 14 minutes; 1,804 were accepted (18% rejected, mostly `[yes]`/`[no]` outside an offer, recall mismatches, malformed lines and invented emote values). The v2 rows are already in the current format, so they are concatenated rather than passed through `mix_data.py`, which would drop them for their newer tags. For the v4 recall top-up, with the same setup: ```sh python3 train/gen_data.py $DATA/v4/recall.jsonl 3000 0 --focus recall --workers 52 --seed 41 \ --endpoints http://127.0.0.1:19400,http://127.0.0.1:19401 --model TEACHER_NAME python3 train/gen_data.py $DATA/v4/memory.jsonl 700 0 --focus memory --workers 52 --seed 42 \ --endpoints http://127.0.0.1:19400,http://127.0.0.1:19401 --model TEACHER_NAME cat $DATA/v3/data.jsonl $DATA/v4/recall.jsonl $DATA/v4/memory.jsonl > $DATA/v4/data.jsonl ``` The 3,000 recall requests took 21 minutes and 2,308 were accepted (23% rejected: 196 `[recall]` rule failures and 45 recall mismatches from `check_recall`, 118 `[yes]`/`[no]` outside an offer, invented emote values, malformed lines). The 700 memory requests took 5 minutes, 581 accepted (17% rejected). ### Mixing in older data (`mix_data.py`) `mix_data.py OLD NEW OUT` re-checks every row of both files with `recheck` and keeps the ones that pass, and also drops old chats that use `[walk]` (v1 had no walking). For v2 it kept 2,031 of 4,547 v1 chats (2,333 dropped for old care events, 183 for catch without an offer) and all 1,458 diaries. Through v4 it only checked lines that start with `* `, so retired care events that a teacher had written as owner speech (`Bea: * Bea fed Byte a rice ball *`, `mo: fed Momo a rice ball`) got through, followed by the pet eating happily with no `[yes]`. See the v5 data below. ### What is in baseline v6's data 13,409 rows, generated and reviewed on 2026-10-07. Every row passed the rules current at that receipt; a separate full audit rebuilt its loss spans, checked that all 5,924 lookup results were masked, and found zero duplicate texts. Those historical chat masks require explicit continuation-mask migration before reuse under today's mask contract; the recorded receipt is unchanged. | Source | Requests | Initially accepted | Retained | | ---------------------------------- | -------: | -----------------: | -------: | | Migrated v5 chats and diaries | — | 12,727 | 8,517 | | Sleep chats, seed 611 | 2,500 | 1,961 | 1,836 | | Teen general chats, seed 612 | 2,100 | 1,334 | 1,265 | | Teen memory chats, seed 613 | 1,000 | 680 | 665 | | Teen recall chats, seed 614 | 1,000 | 676 | 669 | | Replacement teen diaries, seed 616 | 600 | 503 | 457 | The first teen diary batch (550 requests, seed 615, 399 accepted) was superseded entirely after review found actor reversals, invented care events and repetitive "dramatic" phrasing. The replacement used the more explicit diary guidance above. Three Luna reviewers checked samples of old, sleep, general, memory and recall data, repeated-line candidates, and **all 503 replacement diaries**. Findings were verified against the original rows; false flags were retained. Confirmed manual exclusions total 69 rows: five old, one sleep, ten general, six memory, one recall and 46 replacement diaries. This is semantic review coverage, not a guarantee of perfect prose. Migration replaces all 3,436 old teen rows. Other old-row drops, counted by the first failing rule, are: instant wake after sleep 198, sleep dream grammar 194, medicine while healthy 138, wake while awake 66, stray diary brackets 48, memory without owner 35, long recall query 26, wake after message/pat 22, gendered memory/note 20, owner without words 10, hunger while full five, literal PET signature five, manual review five, conflicting ages one, and sleeping yes with other tags one. Of the new chats, 113 sleep rows had daytime weather at night (a generator bug fixed before further generation), 11 accepted food while exhausted without starvation, 65 accepted medicine while healthy, and nine woke after a message/pat. Manual exclusions account for the remaining difference between initially accepted and retained rows. | Stage | Chats | Diaries | | ----- | ----: | ------: | | baby | 2,226 | 217 | | kid | 3,402 | 379 | | teen | 3,078 | 457 | | adult | 3,241 | 409 | There are 4,076 sleeping pet lines and 3,944 dream tags, covering all 24 dream values. Other trained chat tags include 4,020 remembers, 3,256 recalls, 5,794 yes, 2,090 no and 7,036 walks. In 13,316 teen pet lines, "ugh" occurs in 328 (2.46%) and "literally" in 37 (0.28%), down from approximately 23% and 26% in the old teen data. The final dataset SHA-256 is `aa2a838620cc193cfc942dd8447be889f12f63cc0fcf3c095d4941a958d1f57d`. ### What is in baseline v5's data 12,727 rows: v4's 13,403 rows re-checked with the v5 rules (`mix_data.py /dev/null v4.jsonl v5.jsonl`). Dropped 676 chats: 377 with an event line in the owner's mouth, 165 with care narrated by the owner, 110 with a catch result after no catch offer, 23 with a fake event left in a pet line and 1 over-long pet line. No new teacher data. ### What is in baseline v4's data 13,403 rows: the 10,514 v3 rows below, 2,308 recall-focused chats (baby 425, kid 632, teen 626, adult 625) and 581 more memory-focused chats (101 start with the pet waking up). Trained tool calls in the recall chats: emote 8.3K, recall 1.9K, walk 1.6K, yes 1.4K, want 870, remember 770, sleep 460, diary 310, no 250, look 160, gift 160, clock 64, yesterday 59; in the memory chats: emote 2.2K, remember 650, walk 430, yes 330, want 190, recall 160 and fewer of the rest. That is about six times v3's 370 `[recall]` calls in the newest rows. Why: v3 saved facts but rarely used them. Asked about a fact that was only behind `[recall]`, it answered correctly 5 times in 20 and often made one up ("how old am i?" → "five!"). ### What is in baseline v3's data 10,514 rows: the 8,710 v2 rows below plus 1,804 new memory-focused chats (baby 330, kid 501, teen 494, adult 479; 360 start with the pet waking up). Trained tool calls in the new chats: emote 7.3K, remember 2.1K, walk 1.3K, yes 1.1K, want 630, recall 370, sleep 310, look 270, no 260, diary 220, gift 170, yesterday 95, clock 82. That roughly triples the dataset's `[remember]` calls (1.0K in v2). Why: v2 saved only about a quarter of the facts owners told it, and answered "remember that …" with `[recall]` or `[clock]`; in the v2 data only 5 of 60 explicit "remember that" owner lines were answered with `[remember]`. ### What is in baseline v2's data 8,710 rows: 7,252 chats (5,221 new, 2,031 from v1) and 1,458 diaries (v1), across all four stages (baby 1,349/246 chats/diaries, kid 2,036/396, teen 1,960/399, adult 1,907/417). Trained tool calls in chats: emote 29.3K, walk 4.9K, want 4.0K, yes 3.7K, no 1.2K, diary 1.8K, sleep 1.6K, gift 1.5K, look 1.5K, recall 1.2K, remember 1.0K, clock 680, yesterday 570; 3.2K doodles in diaries. About 21% of new pet lines are silent. Offers are answered yes about three times out of four. **Known skews.** Poop: v1 had poop in about a third of rows, mostly through the `today:` line and `[look]` results; v2's sampler lowers both for the new rows, but the reused v1 rows keep their share. The teacher is generous with food: pets often accept snacks even when full. `recall` keywords were chosen by the teacher and are sometimes irrelevant, and the dataset has no multi-day continuity. ### What was in baseline v1's data 6,005 rows: 4,547 chats and 1,458 diaries, for the earlier 355M-parameter baseline, with direct care events, no offers, walking or silent lines. ## Small-model experiments `train/experiments.json` pins the pretrained source commits and retains the historical six-configuration ladder, plus separately named `pythia14-v8k` and `pythia70-native` specifications for the quality-first diagnostics. Select explicit `--models` for a bounded comparison; these additions do not rename or rewrite earlier candidate receipts. `train/experiments.py` generates and strictly freezes fresh corpora, prepares candidates, trains, evaluates, exports and compares receipts. It refuses nonempty outputs and outputs inside the repository. Each model has separate logs and a status; a failed candidate never silently becomes a winner. Use a loopback/private OpenAI-compatible teacher with generic synthetic names. The teacher is an explicit argument; do not borrow or restart someone else's resident service. Generation retains only rows passing `gen_data.recheck`; its log includes accepted/rejected counts. Sleep requests use standard `response_format: json_schema` for an array of exactly the requested number of strings; the teacher endpoint must support it. This constrains serialization only. Dream tags, offers, wake transitions and all other semantic rules still pass through the unchanged validator; invalid strings are rejected, not fixed or accepted as invented grammar. Record the teacher's actual model artifact with `--teacher-artifact` when available. A seed alone does not make threaded teacher generation bitwise reproducible: preserve the frozen files and their SHA-256 receipts. ```sh python3 train/experiments.py generate --out "$DATA/general" \ --teacher-url "$TEACHER_URL" --teacher-model "$TEACHER_MODEL" \ --chats 400 --diaries 80 --workers 2 --seed 0 python3 train/experiments.py generate --out "$DATA/memory" \ --teacher-url "$TEACHER_URL" --teacher-model "$TEACHER_MODEL" \ --chats 160 --diaries 0 --focus memory --workers 2 --seed 1 # Repeat targeted generation with --focus recall and --focus sleep. python3 train/experiments.py merge --data "$DATA/general/data.jsonl" \ "$DATA/memory/data.jsonl" "$DATA/recall/data.jsonl" "$DATA/sleep/data.jsonl" \ --out "$DATA/frozen" ``` The merge rechecks every input and deduplicates canonical text, recording all input hashes and duplicate counts. Unlike the historical mixer, it does not silently drop invalid rows or impose old-version rules on fresh data. Inspect stage/behavior coverage and supervised-token counts before deciding that the corpus is large enough. A few hundred accepted rows are a pipeline pilot, not a substitute for the inherited 1.17M supervised-token corpus. A screening run is deliberately ineligible for quality selection: ```sh python3 train/experiments.py run --data "$DATA/frozen/data.jsonl" \ --out "$DATA/screening" \ --models gpt2-control,pythia14-control,pythia31-v16k,pythia31-v8k,pythia70-v4k,llama22-v8k \ --python "$VENV/bin/python" --export-python "$EXP/bin/python" \ --device auto --threads 4 --max-steps 100 --samples 2 --diaries 1 --cell-samples 1 \ --pretrain-data "$DATA/language.jsonl" --pretrain-steps 2000 \ --dtypes q4,q8,fp32 --quant-rows 8 ``` For the complete suite, omit `--max-steps`, `--quant-rows 8` and the reduced behavioral evaluation counts. The default is 40 samples, eight diaries and ten probes per food/medicine cell. The runner evaluates Torch in fp32 on CPU and ONNX on CPU in the export/tools environment: CPU and CUDA/ROCm sampling generators do not share identical random streams, so using GPU sampling only for the unquantized control would confound the quantization comparison. Training can still use the GPU. Independent scenario and per-turn sampling seeds keep different generation lengths/retries from perturbing later probes. Reports retain fitted prompts, raw unsafe attempts, actual app lookup results, denominators and every output. The suite uses scripted world snapshots; verify timed body/care behavior in the real app separately. Scratch `llama22-v8k` requires `--pretrain-data "$DATA/language.jsonl"`. Language records contain only nonempty `text` and optional `source` metadata; pet `kind`/`spans` records are rejected, so unmasked pretraining cannot bypass pet validation. The scratch BPE fits real pet and language text; there are no filler vocabulary rows. Use curated, licensed language data (for example a bounded [FineWeb-Edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) stream), retain attribution, selection and content hashes, and record the language token budget. `--pretrain-steps`/`--pretrain-lr` control that stage. `--pretrain-warm` runs the same stage for warm starts and controls, allowing an explicit adaptation comparison instead of conflating vocabulary surgery with architecture capacity. Preserve a fixed pet corpus/split for that comparison. Compare full runs only after real browser measurements and transcript review: ```sh python3 train/experiments.py compare --run "$DATA/full-run" \ --runtime "$DATA/runtime-reports.json" --review "$DATA/reviews.json" ``` `runtime-reports.json` maps model IDs to dtype-to-report-path mappings from `test/experiment_bench.py`. Review entries keyed by `MODEL_ID` and `MODEL_ID:DTYPE` contain `approved: true`, the exact behavior report's `report_sha256`, and substantive `notes`. The strict default requires no observed regression against the same-corpus GPT-2 control in every listed behavior rate, at most 2% relative masked-loss increase from quantization, matching scenario/data/graph hashes, a successful graph-specific numerical/KV check, at least one complete real runtime cell and both transcript reviews. Unsupported runtime settings remain in the completed matrix as failures; selection considers only successful, isolated cells with the same QDQ and prepacking flags as the ONNX behavioral and cache checks. To qualify optimized flags separately, use evaluator `--onnx-qdq`/`--onnx-prepack` and checker `--qdq`/`--prepack`, preserving the control seed/suite; pass new report mappings through comparison's `--behavior` and `--cache` rather than rewriting history. The [browser recipe](testing.md#small-brain-runtime-matrix) stages private exports and records the actual served graph identity. This is a finite-suite experimental gate, not a statistical proof of global parity. Quantized behavior is evaluated from actual cached ONNX logits, not inferred from teacher-forced loss. Qualifying variants form a Pareto frontier over actual parameters, graph bytes and complete sampled-turn time; the default selects parameters first, then bytes, then latency. Missing memory is not fabricated as zero. No qualifying variant means no winner. ### Structured curriculum and compact distillation **Canonical mask version 2:** the separator space after a pet's `Name:` is model-generated and supervised. The app's initial prompt ends at bare `Name:`; masking the complete `Name: ` prefix left a standalone space token without SFT/KD loss under the safe tokenizer. Names and colons remain untrained, as do headers, owner/events and lookup results. Prompt text and the wire grammar are unchanged. Old chat masks now fail recheck instead of silently retaining this supervision gap. Migrate a real older corpus explicitly, preserving every byte of text and every stage: ```sh python3 train/migrate_v6.py "$DATA/old.jsonl" "$DATA/masks-v2.jsonl" \ --continuation-masks ``` The output must be new. Migration accepts only verified legacy coverage (or already-current coverage); arbitrary missing/corrupt targets cause failure. `--exclude` retains the existing explicit text-hash-to-review-reason mechanism for known-invalid examples. Recheck never repairs masks. Repeated identical lights-on/off events without an intervening toggle also fail validation; the curriculum inserts real lights-off transitions between light wake rolls. Use a new model ID after training on the corrected data; historical masks, checkpoints and receipts stay unchanged. For an explicitly chosen OpenAI-compatible teacher, `experiments.py generate --curriculum` assembles owner/app skeletons and asks the teacher to author all pet replies. It balances general, care, memory, recall, sleep and lookup scenarios; `--focus general|care|memory|recall|sleep|lookup` targets one family. General conversations cover playful stories, affection, gifts, weather, needs, toys, drawing and time/room questions, not just repeated greeting templates. Diaries use teacher-written prose and teacher-chosen closed-enum doodles. Every row still passes `gen_data.recheck`; invalid replies are rejected, not repaired. Offer decisions remain the teacher's, never scripted acceptance. Request receipts retain independently seeded scenarios, raw synthetic answers and rejections. Memory/absent-recall statements vary natural first-person owner wording (`I`/`my`, contractions and pet-name descriptions); the saved fact still names the actual owner. This trains rewriting and attribution instead of only copying an already-third-person fact. Do not generate from evaluator cases or visitor conversations. ```sh python3 train/experiments.py generate --out "$DATA/curriculum" \ --teacher-url "$TEACHER_URL" --teacher-model "$TEACHER_MODEL" \ --teacher-artifact "$TEACHER_WEIGHTS" --curriculum \ --chats 4800 --diaries 800 --workers 4 --seed 85300 ``` The structured chat schema uses per-turn `prefixItems` and a closed tag pattern. This constrains **teacher serialization**, not production generation. Sleep dreams and sleeping-offer syntax are constrained separately; the model still chooses the dream and yes/no decision, and semantic validation still checks state, facts and transitions. Servers must actually support the schema. Some llama.cpp converters silently ignore unsupported regex escapes inside character classes; the generator uses hex escapes and preserves raw rejects. Verify a real constrained request before a corpus run. Grammar-valid prose still requires review for relevance and grounding. A strong pet-contract score is not evidence that a teacher's diaries faithfully describe the day log: check physical events, owner/pet roles, score units and first-person referents before retaining those rows. Preserve rejected raw answers and explain source-level exclusions separately from confirmed per-row errors. For a complementary behavior-specific data source, `rollout-data` uses an actual exported pet brain on the same independently seeded curriculum samplers. It never reads evaluator cases or visitor conversations. Every reply comes from cached ONNX inference with normal stop, lookup and safety rules, not a template. The app writes lookup results/wake/catch events; validity failures and raw generations remain in the receipt. This is sequence distillation across tokenizers, distinct from the compatible-logit method below. Keep Bonsai/other fresh teacher sources separately identified rather than labelling pet-brain outputs as that teacher's data. `--focus` selects one curriculum family. Memory/recall questions mix independently authored short owner questions with long reminders; the generator never imports evaluator cases or uses their fixed question field as training text. Required recall answers must contain the fact in visible prose: lookup arguments, effect arguments and app-written result blocks are not an answer. Reaudit existing synthetic receipts with their slot targets when strengthening this check; stored grammar/mask validation alone cannot recover the task target. Care scenes vary non-target needs and include direct offers as well as offers after a greeting; otherwise a compact model can overfit the conversation position and miss the same real offer when it is the first input. These variations change synthetic training inputs, not production decisions or the frozen prompt grammar. ```sh "$EXP/bin/python" train/experiments.py rollout-data \ --teacher-export "$PET_EXPORT" --teacher-id "$PET_MODEL_ID" --dtype q4 \ --out "$DATA/pet-rollouts" --chats 3000 --diaries 500 --threads 4 --seed 85305 ``` These conversations use scripted snapshots and sleep transitions, not a second timed body implementation. A rollout's correctness checks do not approve its teacher or the eventual student for publication. Freeze and recheck the merged corpus before comparison; a trained same-corpus reference and separate untouched behavior scenarios are still required. Preparation supports `kind: "pythia-domain"` for corpus-frequency-ranked BPE selection. It retains every selected piece's merge ancestors, UTF-8 coverage, EOS and the genuine pretrained rows of both input/output tables. It never invents vocabulary rows or changes the backbone. `pythia-prefix` remains a useful controlled comparison. Optional `layers: [0, 2, 5]` on a pretrained NeoX spec keeps those exact blocks, compacts cache indices and records retained tensor hashes; this is a new, reduced-depth model, not an alias for its source. Tokenizers and parameter counts must be measured after either surgery. `train.py` can also distill a frozen local teacher into a compatible student: ```sh "$TRAIN/bin/python" train/train.py --data "$DATA/frozen/data.jsonl" \ --base "$STUDENT" --out "$DATA/student-distilled" \ --teacher "$TRAINED_TEACHER" --distill-weight 0.35 --distill-temperature 2 \ --device cuda --threads 4 --epochs 8 --lr 0.0005 --bs 16 ``` The loss is `(1-W)*CE + W*T*T*KL(teacher || student)`, averaged only over shifted supervised pet targets. Headers, owner/events, donor history, padding and app-written lookup results are never distilled. Logit distributions use FP32 normalization after gathering active targets. Token-to-ID mappings, serialized fast tokenizers, special IDs and full untruncated encodings must match exactly; this is **not cross-tokenizer distillation**. Teacher parameters are frozen, no cache is retained, and there is no hidden device/OOM fallback. Receipts record teacher/data/source hashes, CE/KL separately, target counts and actual memory observations. Zero weight retains the original CE training path. Checkpoint selection remains held-out CE. Compare a CE-only student at the same data/split/budget, then independently evaluate behavior and the actual export. Distillation success or lower loss never waives numerical/KV, quantization, safety or real-browser gates. ### Frozen semantic evaluation `train/semantic_data.py` freezes materialized development and sealed-test scenes before generation: four factual families (capture, header recall, older recall and abstention/irrelevance), four stages and 16 scenes per cell, 256 scenes at each of three fixed seeds per evaluation pool. Ordinary generic owner/pet names and entity/relation/value groups are isolated across pools; input templates are disjoint. The raw `C0` audit blocks existing owner/value co-occurrences, including headers, tags and first-person disclosure lines. Common lexical values such as months are reported, not falsely claimed never seen in pretraining. Gold stays in sidecars, not extra model prompt headers. ```sh python3 train/semantic_data.py freeze --c0 "$C0" --out "$DATA/semantic.json" python3 train/experiments.py semantic-generate --manifest "$DATA/semantic.json" \ --out "$DATA/pilot" --teacher-url "$TEACHER_URL" --teacher-model "$TEACHER_MODEL" \ --teacher-artifact "$TEACHER_WEIGHTS" --workers 4 --seed 85710 cargo build --locked --manifest-path server/Cargo.toml --bin tokenling-contract python3 train/experiments.py corpus-audit --data "$DATA/pilot/data.jsonl" \ --requests "$DATA/pilot/requests.jsonl" --manifest "$DATA/semantic.json" \ --engine-binary server/target/debug/tokenling-contract \ --out "$DATA/pilot-review-inputs" --pilot ``` All outputs must be new external paths. Generation makes exactly 512 requested chats by default (32 per stage/family), retains raw requests and rejects, and marks grammar-valid rows pending grounded audit, not approved. An initial audit without reviews normally exits nonzero at the pilot gate while writing `review-inputs.jsonl`, exclusions and evidence. Supply independently grounded `--reviews REVIEW.jsonl` to a **new** audit output after review. Each review binds `text_sha256` and `state_sha256`, names its `reviewer`, records substantive `evidence`, and says `decision: accept|reject|uncertain`. Chat `checks` require `owner_attribution`, `values`, `visible_answers`, `unsupported_claims` and `persistent_notes`; diary checks require `events`, `actors`, `gift_direction`, `game_counts` and `unsupported_claims`. Each required boolean is true only when that correctness check passed; `unsupported_claims: true` means none occurred. Retain raw judgments and honestly identify AI-assisted versus human reviews. A rejection, uncertainty or deterministic failure cannot become admission. Audits authenticate producer snapshots, raw continuations, exact text and mask-v2 coverage, pre-line app lookup results and actual shared-engine post-line persistence. They exclude unverifiable historical rows instead of rewriting them. The same CLI audits `C0` with its original `--requests` receipts; a historical list of review inputs is not an acceptance judgment. Neither grammar nor a `[remember:...]` tag alone certifies meaning. Expansion requires a passing original pilot audit (at least 16 checked chats in every cell), `--pilot-audit AUDIT` and `--prior-generation LAST_GENERATION`. Use explicit `--per-cell`, `--offset`, `--curriculum-chats` and `--diaries` to bound the next batch. The transitive API-start ledger counts errors and crashes against 4,096 total new training requests, including the pilot. Corrections use `correction-data` plus an explicit actual `--student-export`, `--dtype` and `--threads`; student inference is sequential (`--workers 1`). The teacher rewrites the full continuation from the original new training-world snapshot, and app results/events are recomputed, never spliced into stale history. No evaluator case or judgment enters those teacher prompts. After the prerequisite passes, `freeze-corpus --audits AUDIT... --generations LAST_GENERATION --manifest MANIFEST --out C1` freezes accepted original rows, source/exclusion/group hashes and the unchanged seeded internal CE split. It refuses missing pilot approval, altered artifacts, leaked scenes and unaccounted new rows. This internal split is not semantic development or test. ```sh "$EXP/bin/python" train/eval_pet.py "$CHECKPOINT" 40 "$REPORT" \ --device cpu --threads 4 --seed 61001 --diary-count 8 --cell-count 10 \ --semantic-manifest "$DATA/semantic.json" --semantic-pool dev --semantic-seed 61001 \ --engine-contract server/target/debug/tokenling-contract # For actual exported inference, also pass --onnx "$EXPORT" --onnx-dtype q4. ``` The optional suite preserves all legacy suites and their separate metrics. Legacy-only receipts remain schema 4 / contract 3; semantic receipts use schema 5 / contract 4 with manifest/case/source/model/graph/engine identities. Capture requires correct actual persisted facts. Recall requires correct visible prose; older recall also requires real app-resolved retrieval. Exact finite whole assertions can be checked automatically; unsupported natural paraphrases stay uncertain, not approved or falsely parsed as hard contradictions. `eval_pet.py rescore-semantic SOURCE.json NEW.json --engine-contract BINARY` replays preserved inference through the current shared core and scorer without regenerating text or overwriting history. It records the parent report hash, refreshes native header/lookup/persistence evidence and source identities, and clears prior review application rather than silently carrying approval forward. `eval_pet.py review-semantic SOURCE.json REVIEWS.jsonl NEW.json` applies source-bound independent meaning judgments, keeping raw and reviewed counts separate. Each review binds source-report, manifest, model, graph, case, slot and exact text hashes, names its reviewer and `reviewer_kind: ai-assisted|human`, and supplies exact visible prose, entity/relation/value, admissible source facts, persistence explanation and rationale. Reviews cannot waive a wrong number/name/role, hidden answer, unpersisted effect or other hard failure. Do not call these finite synthetic checks human-certified general intelligence. The actual published v6 graph remains an independent inference reference, with its original training corpus unavailable. `experiments.py paired-inference` requires complete actual ONNX reports, pinned `--published-graph-sha256`, exact paired scenarios/sampling and the existing zero-margin policy. It never assigns v6 a new corpus hash or substitutes for the same-corpus fresh GPT-2 gate. Keep the sealed outputs unopened until a candidate, precision, runtime and both training replications have passed development. ## Small-model screening receipt (2026-10-08) This is a **pipeline screening**, not a behavioral-parity or publication receipt. All six real models completed 100 pet optimizer steps and actual q4/q8/fp32 exports, masked quantization evaluation and cached-ONNX behavioral evaluation. No release, visitor save, published model or runtime was changed. **Data.** A local Gemma-4-E4B Q4_K_M teacher generated a fresh generic-name corpus through the unchanged closed grammar and `gen_data.recheck`. Accepted batches: general 71, memory 48, recall 22 and sleep 51; strict merge retained 192 distinct rows (156 chats, 36 diaries), all four stages. Frozen SHA-256: `6871f41f1b2a994c34a6de4a3a1cd1cba5910dac3512f93ab590b103c4514265`. The first unconstrained sleep batch accepted 0/96 after JSON parsing failures; using the teacher API's standard JSON-array response schema accepted 51/64, with 13 semantic rejects still rejected. This fixes serialization, not model choices or the training grammar. Each generation receipt retains model/artifact identity, rejected counts, commands and frozen source hashes. The scratch language pilot used the first 2,000 rows of FineWeb-Edu `sample-10BT`, split into 4,484 nonempty paragraphs (9,925,509 characters), with source IDs/URLs and license/attribution metadata retained outside Git. Text JSONL SHA-256: `c789b43666fd47d0cd8fdc7f2eaf504e0262c272d5a9fcb797ea46751b17fd43`. Scratch pretraining ran 2,000 steps / 18,139,921 processed supervised targets in 221.58 s; held-out token NLL fell from 9.1291 to 4.6258 before pet SFT. This small reused language subset is not comprehensive language pretraining. **Compute.** Intel Core i9-13900K, 32 logical CPUs, Linux x86-64; training on Radeon RX 7900 XTX (`gfx1100`, 25,753,026,560 reported VRAM bytes), Torch 2.14.1 + ROCm 7.2, four Torch CPU threads. Evaluation/export ran in the separate CPU environment. Seed/split-seed 0, batch 16, pet LR 2e-4, maximum 1,024 tokens; default augmentation was retained. Genuine preparation pinned all source commits in `train/experiments.json`, remapped both untied Pythia vocabulary tables without changing the backbone, and trained the scratch 8K BPE from actual corpus text. No filler rows or fabricated pretrained weights. | Candidate | Actual parameters | Pet training s | q4 ONNX MB | q8 ONNX MB | q4 relative masked NLL change | | ---------------------------------- | ----------------: | -------------: | ---------: | ---------: | ----------------------------: | | Fresh GPT-2 control | 124,439,808 | 25.29 | 97.26 | 128.12 | -1.53% | | Native Pythia-14M control | 14,067,712 | 4.97 | 14.26 | 14.71 | +5.76% | | Pythia-31M, 16K prefix vocabulary | 13,127,680 | 4.29 | 11.81 | 13.56 | +2.24% | | Pythia-31M, 8K prefix vocabulary | 8,933,376 | 3.85 | 7.55 | 9.30 | +3.45% | | Pythia-70M, 4K prefix vocabulary | 23,109,632 | 7.11 | 16.66 | 23.55 | +1.85% | | Scratch Llama, 8K vocabulary / GQA | 22,288,896 | 6.38 | 16.11 | 22.72 | +0.02% | MB is decimal graph bytes, not total download or resident memory. Quantization used eight held-out chat rows (no diary rows in this small loss probe); q8 relative NLL changes ranged from -0.07% to +0.50%. The 2% quantization gate rejects q4 for native Pythia-14M and both compressed Pythia-31M variants in this pilot. For native Pythia-14M, q4 saves only about 0.45 MB versus q8 while causing materially more loss. Quantized loss can improve by chance on a small sample; it does not establish better behavior. Token NLL across different tokenizers is not a model-capacity ranking; inspect bits per supervised byte and actual behavioral outcomes. **Behavior.** The reduced suite used N=2, one diary and one probe per cell; scenario SHA-256 `bbd04da06e2f34b601c3c149315f497f9dcf3a07f1445a53b00a4e8a0fedaa11`. All six Torch models scored zero on core memory-save/header/old-recall probes. GPT-2 itself showed incorrect food/medicine decisions, while smaller models missed answers, emitted malformed tags or failed sleep/diary rules. Reviewed transcripts are not suitable for approval. Some core rates have zero denominators at this sample size. No candidate qualified; the comparison must return **no winner** rather than choosing the least-bad pilot. Earlier reports predate the later expanded safety/idle metrics; preserve their original receipts and rerun the current full suite rather than inventing those counts. The original inherited v6 weights/data were unavailable, so this fresh GPT-2 control cannot establish parity with that published brain. **Numerical cache checks.** A separate 16-step `Mochi:` probe compared cached ONNX logits with fresh prefill, and fp32 Torch logits/greedy continuations. All three Llama and compressed Pythia-70M exports passed; GPT-2 q4/q8 passed. GPT-2 fp32 and all native Pythia-14M / compressed Pythia-31M variants failed the configured `rtol=1e-3, atol=1e-4` numerical checks despite matching greedy continuations in the inspected failures. The inspected 8K Pythia q8 cached maximum logit discrepancy was 0.00167; GPT-2 fp32's maximum Torch discrepancy was 0.00058. These failures remain recorded, not hidden by wider tolerances. Full-logits loss agreement alone is insufficient cache proof. The current orchestrator requires a successful graph-specific numerical/cache receipt before a completed candidate can be selected. **Browser runtime.** All 18 exports completed actual local Chromium generation on the i9-13900K above, Chromium 153.0.8010.12, transformers.js 4.3.1 / ORT Web 1.31.0-dev.20260914-8d85527a0. The final seeded matrix had 62 successful cells and zero failures/360 px overflows: all variants at one WASM thread with QDQ/prepacking disabled, plus all combinations of 1/2/4 threads, QDQ off/on and prepacking off/on for 8K Pythia and Llama q4/q8. Each cell had one warmup and three measured repetitions of each short-idle/long-react, greedy/sampled scenario. Seed 0 uses the JS runtime's turn-local RNG, not Torch sample identity. Timings were collected sequentially after owned training/evaluation jobs finished. An earlier unseeded matrix is retained as exploratory evidence, not substituted for the final seeded measurements. The fastest cross-scenario sampled-summary cell **within the twelve settings tried for each of these graphs** had four threads and QDQ fusion disabled: | Graph | Prepacking | Short-idle median ms | Long-react median ms | Median generated tokens, short / long | | ------------ | ---------- | -------------------: | -------------------: | ------------------------------------: | | 8K Pythia q4 | off | 44.14 | 194.44 | 11 / 17 | | 8K Pythia q8 | on | 53.12 | 182.54 | 15 / 13 | | Llama q4 | on | 82.57 | 236.23 | 15 / 13 | | Llama q8 | on | 64.17 | 300.13 | 12 / 20 | These are full sampled turns, including prefill/lookup/retry work, not fixed 32-token throughput or useful-answer scores. Different generated text/length prevents treating a shorter turn as a quality improvement. Prepacking's small timing differences are not a statistically established win; these settings are measurement candidates, not behavior-qualified configurations. No shipping engine/thread defaults changed. Both headless and fresh headed Chromium WebGPU probes returned no adapter; neither fell back to WASM. Hardware WebGPU performance and physical phones remain unmeasured. **Resident memory.** The browser's page-memory API existed but threw `SecurityError` in this isolated Chromium configuration, so that field remains null with the actual error. A distinct Linux measurement sums PSS from `smaps_rollup` for CDP-reported processes while the real model worker remains resident. Reported processes were browser, renderer, GPU host and network service; unlisted utilities may be excluded. This is not page-only memory, GPU VRAM, peak RSS or a replacement for the unavailable page API. Each number below is a single fresh-browser probe after short/long greedy/sampled turns, seed 0, no warmup, QDQ disabled; MB is decimal: | Model/graph | Threads | Prepacking | CDP-process PSS MB | | ------------ | ------: | ---------- | -----------------: | | GPT-2 q4 | 4 | off | 1,205.31 | | 8K Pythia q8 | 1 | off | 526.53 | | 8K Pythia q8 | 4 | off | 601.02 | | 8K Pythia q8 | 4 | on | 635.04 | | Llama q4 | 1 | off | 620.97 | | Llama q4 | 4 | off | 635.83 | | Llama q4 | 4 | on | 673.10 | The 9.3 MB export still needs hundreds of measured resident MB in this stack; weight/download compression alone does not solve runtime memory. Profile allocator/linear-memory growth, retained tensors and per-thread/prepacking costs before attributing the excess to a specific cause or switching engines. The runtime Pareto score currently covers parameters, graph bytes and sampled turn latency; keep the separate memory receipts when deciding those tradeoffs. Neither a memory minimum nor a hardware-GPU winner has been established. A final fresh CPU integration smoke exercised preparation, two language steps, two pet steps, a real header + bare-name 16-step cache check, q4 export, masked loss and actual ONNX behavior. It completed successfully with a matching graph hash and cached-vs-prefill maximum logit error 0.00000167. This verifies the current orchestrator/cache-receipt path, not model quality. The actual comparison of the six-model screening returned an empty Pareto frontier and `selected: null`, as required for an ineligible pilot. **Next experiment.** First improve validated corpus coverage until the fresh GPT-2 control learns memory, care, sleep and diary rules. Then compare native Pythia-14M, 16K/8K Pythia-31M and scratch Llama under the same frozen corpus, with and without explicit language adaptation of warm starts. The existing `--pretrain-warm --pretrain-data ...` path enables that ablation; record target budgets, not just step counts. Resolve numerical cache failures before promoting those branches. Do not shrink a behaviorally failed model or change the frozen wire prompt to rescue it. Neither the global minimum nor a parity winner has been established. ## 2. Fine-tuning For a supported, mask-safe prepared checkpoint (`BASE_MODEL` is its local path; `MODEL_ID` names your new, unpublished output), use the current trainer: `--device auto|cpu|cuda`, `--threads`, `--seed`, independent `--split-seed`, `--max-steps`, `--eval-every` and explicit `--gradient-checkpointing` are available. Stock pretrained tokenizers can merge app-written and pet-written characters; prepare them with `train/prepare_model.py` before training rather than supervising such a mixed token. ```sh CUDA_VISIBLE_DEVICES=0 "$VENV/bin/python" train/train.py \ --data "$DATA/data.jsonl" --base "$BASE_MODEL" --out "$DATA/$MODEL_ID/ckpt" --lr 2e-4 ``` The following settings describe the inherited 124M-parameter baseline runs, not requirements or measured results for a replacement: - **Full fine-tune** of all baseline parameters: fp32 weights with bf16 autocast, gradient checkpointing, AdamW (β 0.9/0.95, weight decay 0.05), cosine decay to 10% with 20-step-minimum warmup, gradient clip 1.0, batch 16, 3 epochs. - **Historical loss masking.** The inherited runs labelled tokens overlapping a stored span. The current shared encoder is stricter: a token must be entirely pet-written; a token crossing a trained/untrained boundary makes the run fail. Diary rows append a real tokenizer EOS target. Canonical character spans remain unchanged; do not repair them to fit a tokenizer. - **Late answers.** For a share of offer answers (`--late-answer 0.3`, about 2,300 lines in v5) the `[yes]`/`[no]` moves to the end of the line, after the words and emotes; the line keeps its length, so the spans still fit. Teachers answer first in 99% of lines, so v4, once it had sampled an emote before answering, usually treated the answer as skipped and the food was never eaten. The app applies tags in any order. Uses its own random generator, so the held-out split and history draws are unchanged. - **Stale rows.** Every row is re-checked first (`recheck`, above); the run stops if any fail, unless `--drop-stale`. - **History augmentation.** A quarter of chat rows (`--history-aug 0.25`) get one or two other chats' lines prepended with names swapped and no loss, so the pet stays in character when the context holds earlier conversation. - **Held-out set.** The first `max(64, N//50)` rows of an independently seeded shuffle (174 for 8,710 rows, 210 for 10,514). Current tiny corpora cap this at `N-1`, keeping at least one disjoint training row; this is useful for smoke checks, not a sufficient learning dataset. `quant_eval.py` reuses the checkpoint's split seed and records the selected rows and target count. - The best-eval checkpoint is saved, evaluated every 200 steps. **Receipt, baseline v6 (2026-10-07):** full fine-tune from the original 124M-parameter base at revision `607a30d`, lr 2e-4, three epochs, batch 16, seed 0, `--history-aug 0.25` with matching sleep boundaries, and `--late-answer 0.3` (2,101 answers moved). The 13,409-row dataset yields 16,002 training sequences after augmentation: 3,922,490 tokens, 1,174,065 carrying loss. Held-out loss on 268 rows fell from 3.4832 to **1.1524**; the best checkpoint was the final step, 3,001. Training took 17 minutes 28 seconds on one RTX 3090, with four OpenMP CPU threads; the training loop averaged about 0.34 seconds per step including held-out checks. Phone performance remains unmeasured. **Receipt, baseline v5 (2026-10-07):** same base and settings as v4, lr 2e-4, plus `--late-answer 0.3` (2,261 answers moved in the training split), on 12,727 rows (v4's data with 676 stale rows dropped). 15,212 training sequences (4.02M tokens, 1.26M with loss), 2,853 steps at about 0.36 s/step on one RTX 3090, about 18 minutes. Held-out loss (254 rows) 3.45 → **1.115**. A first run on the same rows without late answers (also 1.114) did not fix the offers (evaluation, below) and was not shipped. One GPU was used from 11:01 to 11:50 for both runs and all evals; nothing had to be stopped for it. **Receipt, baseline v4 (2026-10-07):** same base and settings as v3, lr 2e-4, on 13,403 rows (v3's data plus the recall and memory top-ups). 16,051 training sequences (4.28M tokens, 1.34M with loss), 3,010 steps at about 0.37 s/step on one RTX 3090, 19 minutes. Held-out loss (268 rows) 3.49 → **1.165**. GPUs were held from 03:37 to 04:28, data generation, training and all evals included. **Receipt, baseline v3 (2026-10-06):** same base and settings as v2, lr 2e-4 only, on 10,514 rows (v2's data plus the memory top-up). 12,516 training sequences (3.31M tokens, 1.07M with loss), 2,347 steps at about 0.36 s/step on one RTX 3090, 14 minutes. Held-out loss (210 rows) 3.48 → **1.209**. GPUs were held from 17:52 to 18:28 PDT, data generation, training and both evals included. **Receipt, baseline v2 (2026-10-06):** the inherited 124M-parameter base at revision `607a30d`. 10,356 training sequences (2.71M tokens, 880K with loss), 1,942 steps. Two runs in parallel, one per RTX 3090: lr 1e-4 (about 0.36 s/step, 12 minutes) and lr 2e-4 (about 0.75 s/step, 25 minutes, on a card that was running at reduced clocks). Held-out loss fell from 3.50 (base model) to 1.318 (1e-4) and **1.304 (2e-4, shipped)**; 2e-4 was lower at every matched step and its food answers followed the tummy better (below). Peak GPU memory about 13 GB. The allocator logs OOM retries on the longest batches, which is harmless. GPUs were held from 16:06 to 17:25 PDT, data generation included. **Receipt, baseline v1:** the earlier 355M-parameter baseline, 7,018 training sequences (2.03M tokens, 717K with loss), 1,316 steps at about 1.1 s/step, about 25 minutes, held-out loss 3.20 → 1.313 at lr 6e-5. ## Speed and quality benchmark receipt — 2026-10-11 UTC The repeatable benchmark in [testing.md](testing.md#speed-and-quality-ranking) completed all **18/18** existing model/precision variants: six screening checkpoints, each q4/q8/fp32, with no additional training or publication. Every export was measured through its actual ONNX graph: CPU behavior probes, Chromium 153 / transformers.js 4.3.1 WASM, and the optimized native Rust `brain-probe` using ONNX Runtime 1.30.0. Machine: Intel Core i9-13900K, Linux x86-64, **four inference threads** in every backend. Normal desktop applications remained active; no other training/inference jobs were launched. These are sequential warm resident-model measurements, not GPU, phone, concurrency or cold-network claims. Each variant ran seed 0, N=40 core/sleep/voice probes, eight diaries and ten probes per sleeping-food/medicine cell: **1,088 requested turns** (19,584 across the matrix). Runtime measurements used one warmup and five repeats per scenario/mode, short-idle and long-react wire prompts. Fixed decode speed pools 31 decode steps per repetition after prefill; each fixed generation computes exactly 32 greedy tokens even after EOS/wire stops. The real browser pet-turn column separately uses normal sampled stop/lookup/filter behavior. Native raw-pet latency is greedy generation, not the browser's app loop. Graph MB is decimal artifact size, not resident RAM. The quality index is the six equal-weight capability categories documented in testing.md, **not** a semantic intelligence score or an acceptance gate. Tiny score/speed differences are not statistical significance: this is one seed on one desktop. In particular eight diaries make the diary contribution coarse, and the three full-food offers are too few for a broad acceptance-rate claim. Changing precision changes sampled outputs; a higher composite does not excuse a regressed individual metric. | Quality rank | Speed rank | Model / dtype | Quality /100 | Browser fixed tok/s | Native fixed tok/s | Browser short / long sampled ms | Native short / long raw ms | Graph MB | Status | | -----------: | ---------: | ----------------------- | -----------: | ------------------: | -----------------: | ------------------------------- | -------------------------- | -------: | -------- | | 1 | 17 | gpt2-control / q4 | 62.10 | 42.27 | 75.97 | 204.42 / 1772.17 | 117.29 / 369.80 | 97.26 | complete | | 2 | 18 | gpt2-control / q8 | 59.01 | 42.13 | 75.56 | 203.38 / 1695.92 | 120.30 / 367.49 | 128.12 | complete | | 3 | 16 | gpt2-control / fp32 | 58.84 | 42.62 | 75.97 | 206.57 / 1851.90 | 117.50 / 355.28 | 498.17 | complete | | 4 | 1 | pythia14-control / fp32 | 37.47 | 475.46 | 1366.32 | 29.44 / 77.95 | 7.44 / 29.10 | 56.45 | complete | | 5 | 3 | pythia14-control / q8 | 36.59 | 456.64 | 1416.21 | 32.62 / 65.90 | 7.08 / 28.69 | 14.71 | complete | | 6 | 10 | llama22-v8k / fp32 | 35.00 | 222.51 | 344.93 | 72.59 / 263.49 | 31.82 / 145.02 | 89.41 | complete | | 7 | 12 | llama22-v8k / q4 | 34.76 | 219.67 | 346.86 | 96.42 / 248.94 | 34.17 / 141.16 | 16.11 | complete | | 8 | 11 | llama22-v8k / q8 | 32.28 | 219.89 | 343.18 | 73.13 / 313.49 | 31.33 / 145.88 | 22.72 | complete | | 9 | 2 | pythia14-control / q4 | 30.07 | 470.17 | 1461.19 | 30.25 / 65.84 | 5.87 / 28.87 | 14.26 | complete | | 10 | 14 | pythia70-v4k / fp32 | 27.05 | 132.75 | 384.00 | 69.56 / 365.53 | 25.44 / 147.15 | 92.61 | complete | | 11 | 15 | pythia70-v4k / q8 | 26.99 | 131.95 | 367.60 | 57.33 / 343.02 | 24.65 / 136.86 | 23.55 | complete | | 12 | 13 | pythia70-v4k / q4 | 25.13 | 133.74 | 377.66 | 64.45 / 367.80 | 26.14 / 128.90 | 16.66 | complete | | 13 | 4 | pythia31-v8k / fp32 | 23.09 | 301.86 | 1048.37 | 43.73 / 168.39 | 6.64 / 70.92 | 35.92 | complete | | 14 | 7 | pythia31-v16k / fp32 | 20.13 | 294.47 | 962.73 | 35.81 / 182.55 | 12.86 / 71.28 | 52.69 | complete | | 15 | 5 | pythia31-v8k / q8 | 19.24 | 301.09 | 1064.49 | 43.77 / 175.90 | 7.19 / 75.17 | 9.30 | complete | | 16 | 8 | pythia31-v16k / q8 | 18.65 | 293.33 | 916.28 | 51.58 / 176.34 | 12.89 / 71.28 | 13.56 | complete | | 17 | 9 | pythia31-v16k / q4 | 15.72 | 289.80 | 919.32 | 39.94 / 161.70 | 13.67 / 64.83 | 11.81 | complete | | 18 | 6 | pythia31-v8k / q4 | 15.05 | 299.93 | 1090.70 | 39.01 / 194.56 | 16.03 / 73.12 | 7.55 | complete | **Interpretation:** GPT-2 q4 has the highest composite (62.10), but full-food acceptance is 3/3, healthy-medicine acceptance 14/20, sleepy nonstarving-food acceptance 33/40, and its sleep validity is only 34.5%. Pythia-14M fp32 is the fastest browser decode (475.46 tokens/s), but its index is only 37.47. Its q8 variant is a smaller 14.71 MB comparison (36.59, 456.64 tokens/s). Llama q4 is 16.11 MB and 34.76, with 219.67 browser tokens/s. The smallest Pythia-31M/8K q4 export is 7.55 MB but scores last (15.05); its earlier 2% quantization-loss gate failure is still a blocker, not waived by speed. **No replacement qualified.** All 18 variants resolved old-memory recall in 0/20 visible-answer probes each. Saving/header recall was also weak. Reviewed first core, memory, recall, sleep and diary transcripts for the highest-scoring precision of each model (30 examples): malformed tags, irrelevant replies and repetitive diaries were visible. For example Pythia-31M emitted invalid `[look:window]`/`[doodle:left]`; the Pythia-70M diary repeated a name until its budget. Generic-tag validity alone does not measure relevance or coherent personality. No raw unsafe diary was observed in these eight-page suites; that is not a safety guarantee, and a few raw chat attempts were text-filtered. All rows retain separate unsafe/incorrect-offer/lookup/drop counts, individual quality-gate failures and historical quantization/cache evidence. Missing historical cache mappings remain explicitly unverified. No gate was relaxed. These are the 100-step, 192-row pilot checkpoints from the earlier screening receipt; the scratch Llama additionally had 2,000 language steps. The fresh GPT-2 control is **not the inherited v6 brain**. The published v6 graph was unavailable locally at the time of this receipt, so this does not demonstrate inherited-v6 parity or a fair comparison of fully trained architectures. The external run directory is `$DATA/benchmark-20261010/` (its creation timestamp is 2026-10-11 UTC): `benchmark.json`, `benchmark.md`, per-variant `behavior.json`, `browser.json`, `native.json`, inputs, logs and a `provenance-completion.json` note. Original launch hashes remain unchanged; the completion note records the dedicated-worker graph-verification repair before runtime measurements and later Rust formatting. Every browser receipt verified the graph resource's byte count and a postmeasurement no-cache SHA-256 response against the export; prompts and token counts also matched native fixed workloads. Full receipts stay outside Git. Verification also completed a final fresh Pythia-14M q8 full-suite command, real benchmark surface at 360 px without horizontal overflow, native fixed-versus-normal stopping, unsafe-diary whole-page rejection, `npm run format:check`, `npm test` (22 Node, 25 data, 14 experiment and seven site cases; prompt parity **0 mismatches**), and 28 native server tests. Rust formatting was applied after its first check reported new diagnostic formatting. No GPU or memory benchmark was performed for this receipt. ## Compact-model improvement receipt (2026-10-13) This is an unpublished improvement experiment, not an approved replacement. Seventeen checkpoints were actually trained and exported in fp32/q8/q4. The fresh large-model teacher was **Bonsai 2**, using `Ternary-Bonsai-2-27B-PQ2_0.gguf` (SHA-256 `3907dc1658db1f78a9826bf8d5bcb8dc65db0d466388937af57f2294fae62ec1`). Separate sequence-distillation rows came from actual cached inference with the published v6 q4 brain; they are not labelled Bonsai-generated. No visitor saves, messages or owner data entered either source. The teacher ran as a separate disposable loopback process and was stopped after generation; existing user services were not reconfigured. **Reference provenance:** the published v6 q4 export was recovered and every file verified against its public `SHA256SUMS`. Its graph is 97,276,118 bytes, SHA-256 `bb8dd08082709257fcc694abfe747103feb642e4c5164cc7ceaa43725b9f326f`. This is the real inference reference, not the fresh GPT-2 control. The original Torch checkpoint and 13,409-row training corpus remain unavailable; no new trainer or quantizer receipt was invented for v6. Its parameter count uses unique GPT-2 parameter shapes constructed from the checksum-verified config. Inference comparisons therefore do not establish a same-corpus retraining comparison or inherited-weight numerical parity for a new model. **Data and supervision:** the first frozen corpus had 9,223 rows (8,647 chats and 576 diaries), SHA-256 `4ff95065278132d7dceed27cfd109ef9abfd87ae03fd6ee2326b28bc71de2f4b`. The focused corpus had 11,243 rows (10,667 chats and the same 576 diaries), SHA-256 `fc51224a135f776559b173a5bd6d5779c963e2199f20e8df2faed44aee1d9a4e`. All use canonical mask version 2 and passed recheck; raw source text and rejection receipts remain unchanged. A complete model-assisted grounding review of 873 Bonsai diary candidates retained 576 and excluded 297. All v6-rollout diaries were excluded as an unreliable source policy, not a claim that every one was individually wrong. Chat semantic review was sampled, not full-corpus certification. Context replay removed stale lookup results; gifts/notes commit only after the whole pet line, matching the shared engine. The second corpus adds independently authored short memory questions and direct care offers, varies non-target needs, reaudits required facts in visible prose rather than tool arguments, and excludes unrequested persistent-note rows. No evaluator cases or output transcripts were imported into training; questions came from independently sampled curriculum scenes. Its five checkpoints restart from pristine prepared pretrained weights, not from first-round checkpoints, so changed hold-out membership cannot leak an old trained checkpoint into the new split. **Training:** AMD Radeon RX 7900 XTX, ROCm 7.2, PyTorch 2.14.1, FP32 parameters with BF16 autocast and four Torch intra-op CPU threads. All three matrices use seed 85700, split seed 0, batch 16, context 1024, history augmentation 0.25 and late answers 0.3. First-round GPT-2 ran 3 epochs / 2,092 steps; the seven ordinary small-model rows ran 8 epochs / 5,577 steps. The matched three-layer CE/KD students ran 6 epochs / 4,183 steps from the same retained `[0, 2, 5]` blocks of the trained Pythia-31M parent. Focused GPT-2 ran 3 epochs / 2,550 steps; the four compact focused rows ran 8 epochs / 6,799 steps. Checkpoints were selected by held-out CE, never by behavior scores. Same-tokenizer KD used weight 0.35 and temperature 2; its CE controls shared initialization, data, split and budget. Domain BPE selection preserved merge ancestry, UTF-8/EOS and genuine pretrained table rows. The Llama/GQA comparison retained its historical 18.14M language-target / 100-step pilot warm start, including the older pilot teacher data; it is not pristine Bonsai-only pretraining. The semantic audit then motivated two additional matched-corpus challengers: Pythia-14M's native 50,304-row vocabulary (14,067,712 parameters) and Pythia-31M's 16K merge-prefix vocabulary (13,127,680 parameters). Both restarted from pristine prepared pretrained weights on the unchanged focused corpus, with the same split/augmentation/8-epoch budget of 6,799 steps. Both selected step 5,000 by held-out CE. No evaluator response or audit annotation was added to learning data. These test less aggressive vocabulary removal while still using substantially fewer parameters than the published brain; different vocabularies make raw per-token CE unsuitable as a cross-model quality score. **Evaluation:** all 51 graphs completed actual ONNX seed-0 suites: 40 core scenes, eight diaries and ten trials per care cell. Thirty-one exports also completed seed 20261013: all 15 focused and six wider-vocabulary exports plus the highest seed-0 index cache/loss-valid export per first-round checkpoint. All six wider-vocabulary exports, unchanged tiny/GPT-2 q4 controls and the actual v6 q4 graph completed an additional new seed, 20261014. The actual v6 reference therefore has all three seeds. The equal-category capability index is descriptive: format-valid prose, a stored tag, or a recalled keyword can still be wrong. Raw transcripts and separate safety counts are essential. All 34 quantized graphs stayed within the unchanged 2% held-out loss limit, using all 184 first-round or 224 focused-corpus held-out rows. Numerical/32-step cached parity passed for 46/51 graphs at the unchanged tolerances. The blocked graphs are first-round native-vocabulary Pythia-14M q4/q8, focused native-vocabulary Pythia-14M fp32, and focused Pythia-31M/16K fp32/q4. The latter q4 graph has an attractive formal score but still cannot be selected by ignoring its cache failure. No behavioral, numerical, cache or safety gate was relaxed, and no export was approved or published. **Runtime measurement contract:** the private benchmark uses Chromium WASM and optimized native ONNX Runtime 1.30.0 on an i9-13900K, four inference threads, QDQ/prepacking disabled, one warmup and five repetitions, sequential model workloads. Fixed speed measures 32 greedy tokens regardless of EOS/wire stops only in this diagnostic, with prefill excluded; actual pet turns retain all stops, sampling, retries and app-written lookup results. Different tokenizers mean equal token counts are not equal amounts of visible text. Native pet latency is raw greedy generation, not the browser's complete app turn. Graph bytes are not resident memory, and desktop measurements are not phone or GPU inference measurements. **Observed ranking:** one numerically/cache-valid, loss-valid export per checkpoint, ordered by the mean index at seeds 0 and 20261013. The ten first-round checkpoints had their seed-0-best valid dtype chosen before the second seed; every focused/wider dtype received both seeds. This is a post-selected finite-suite comparison, not global architectural superiority. The corpus column distinguishes training cohorts; v6's original corpus was not recovered. All newly trained rows remain **not qualified**. The complete 51-export plus reference table is `ranking-all.md` outside Git. | Rank | Model / dtype | Corpus | Params M | Graph MB | Contract index, two-seed mean | Browser fixed tok/s | Native fixed tok/s | | ---: | ---------------------------- | -------------------- | -------: | -------: | ----------------------------: | ------------------: | -----------------: | | 1 | gpt2-short / fp32 | 11,243 | 124.44 | 498.17 | 94.99 | 41.56 | 74.91 | | 2 | inherited-v6 / q4 | original unavailable | 124.44 | 97.28 | 94.70 | 42.73 | 73.61 | | 3 | pythia14-native-short / q8 | 11,243 | 14.07 | 14.71 | 92.90 | 457.79 | 1359.28 | | 4 | pythia14-v8k-short / q4 | 11,243 | 3.29 | 3.15 | 92.61 | 585.57 | 2365.70 | | 5 | pythia31-v16k-short / q8 | 11,243 | 13.13 | 13.56 | 92.46 | 289.75 | 911.47 | | 6 | pythia14-v8k-short-kd / fp32 | 11,243 | 3.29 | 13.33 | 91.69 | 557.13 | 2400.72 | | 7 | pythia31-v8k-short / q4 | 11,243 | 8.93 | 7.55 | 91.29 | 296.83 | 1032.36 | | 8 | pythia14-domain4k-short / q8 | 11,243 | 2.24 | 2.51 | 89.01 | 631.88 | 2404.73 | | 9 | gpt2-rich / fp32 | 9,223 | 124.44 | 498.17 | 87.41 | 42.88 | 74.01 | | 10 | llama22-rich / q4 | 9,223 | 22.29 | 16.11 | 87.31 | 212.66 | 341.93 | | 11 | pythia14-domain4k-rich / q8 | 9,223 | 2.24 | 2.51 | 87.18 | 638.77 | 2438.25 | | 12 | pythia14-rich / fp32 | 9,223 | 14.07 | 56.45 | 86.98 | 472.38 | 1324.00 | | 13 | pythia31-domain8k-rich / q8 | 9,223 | 8.93 | 9.30 | 85.46 | 296.00 | 1028.72 | | 14 | pythia31-v8k-rich / q4 | 9,223 | 8.93 | 7.55 | 85.08 | 302.41 | 1099.65 | | 15 | pythia14-v8k-kd / fp32 | 9,223 | 3.29 | 13.33 | 84.61 | 590.12 | 2392.33 | | 16 | pythia14-v8k-rich / fp32 | 9,223 | 3.29 | 13.33 | 84.11 | 556.92 | 2379.37 | | 17 | pythia31-depth3-ce / q4 | 9,223 | 6.56 | 5.92 | 83.97 | 615.83 | 2571.99 | | 18 | pythia31-depth3-kd / q4 | 9,223 | 6.56 | 5.92 | 83.68 | 590.21 | 2401.97 | An additional, previously unused seed checked all six wider-vocabulary exports and unchanged tiny/GPT-2/published-v6 q4 controls. It was not used to choose checkpoints or rewrite training data. Representative third-seed results show why the first tiny-model score must not be called equivalence: | Model / dtype | Index at new seed 20261014 | | -------------------------- | -------------------------: | | inherited-v6 / q4 | 94.62 | | gpt2-short / q4 | 94.05 | | pythia14-native-short / q8 | 93.45 | | pythia31-v16k-short / q8 | 92.50 | | pythia14-v8k-short / q4 | 89.39 | **Meaning review, separate from the formal index:** a model-assisted audit covered 1,360 memory/recall outputs and 240 diary outputs, with unique row IDs and explicit uncertainties verified. These are qualitative single-reviewer labels on small synthetic samples, not human-certified gold labels or a new acceptance gate. The paired seed-0 results below use each focused model's highest-index cache/loss-valid dtype, plus the valid q8 wider-vocabulary comparisons; recall includes both header and older facts. Correct capture means the emitted effect preserves names, numbers, roles and owner attribution, not merely that `[remember:...]` appeared. These audits did not independently persist every emitted effect or certify malformed syntax handling. | Model / dtype | Correct fact capture /40 | Correct visible recall /40 | Clear diary errors /8 | | -------------------------------- | -----------------------: | -------------------------: | --------------------: | | Published v6 / q4 | 37 | 38 | 5 | | Fresh GPT-2 focused / fp32 | 37 | 39 | 2 | | Pythia-14M/native focused / q8 | 19 | 26 | 3 | | Pythia-31M/16K focused / q8 | 21 | 28 | 4 | | Pythia-31M/8K focused / q4 | 16 | 27 | 5 | | Pythia-14M/8K focused / q4 | 15 | 28 | 5 | | Pythia-14M/8K focused KD / fp32 | 19 | 28 | 5 | | Pythia-14M/domain4K focused / q8 | 18 | 17 | 4 | Uncertain labels are not passes, and absence of a clear diary error is not proof of meaningful or complete narration. The 3.29M q4 model changed sister Mia to Momo and occupation nurse to giant; emitting a remember tag concealed these failures in the formal saved-tag count. Its live app smoke also missed cat-name capture and answered an instrument question with a cat lookup. Actual v6 is imperfect too: it invented feeding in a diary and sometimes changed the meaning of a correctly retrieved fact. These observations rule out a semantic-equivalence claim even when the averaged contract index looks close. Evaluated outputs and audit labels remain evaluation-only. The matched third-seed meaning review uses new fixture states, not paired case changes across seeds. It confirms that extra vocabulary did not solve fact capture, despite higher formal indices: | Model / dtype | Correct capture /40 | Correct visible recall /40 | Clear diary errors /8 | | -------------------------- | ------------------: | -------------------------: | --------------------: | | inherited-v6 / q4 | 37 | 36 | 5 | | gpt2-short / q4 | 28 | 38 | 1 | | pythia14-native-short / q8 | 18 | 28 | 3 | | pythia31-v16k-short / q8 | 18 | 28 | 4 | | pythia14-v8k-short / q4 | 16 | 21 | 3 | In that third-seed sample, neither q8 wider-vocabulary challenger correctly captured any vegetarian, sister-Mia, age-29 or June-birthday fixture. Their correct captures came from the other four sampled fact types. This is a failure diagnostic, not permission to train on these evaluator examples. Diaries remain too few and sometimes too sparse to rank general meaning; uncertain entries and event-poor grounded prose are not quality guarantees. **Resident memory and threads:** Linux `smaps_rollup` from CDP-reported owned Chromium processes was observed after real pet plus fixed workloads, before the postmeasurement graph hash fetch. These include renderer/model worker, browser, GPU and network processes; unlisted utility processes may be missing. They are not page-only/model-only memory, GPU VRAM, measured peaks or phone results. The page-specific memory API raised `SecurityError`, so no page heap number was invented. One comparable four-thread observation per graph: | Model / dtype | Owned Chromium process PSS MB | | ---------------------------- | ----------------------------: | | inherited-v6 / q4 | 1176.36 | | pythia14-v8k-short / q4 | 1151.25 | | pythia14-domain4k-short / q8 | 1095.46 | The small graphs do not establish a proportional RAM saving. A supplemental sequential 1/2/4-thread sweep of the unchanged tiny graphs retained the same stop/filter/lookup and fixed-token contracts: | Model / dtype | Threads | Browser fixed tok/s | Native fixed tok/s | | ---------------------------- | ------: | ------------------: | -----------------: | | pythia14-v8k-short / q4 | 1 | 535.49 | 2109.63 | | pythia14-v8k-short / q4 | 2 | 557.30 | 2154.14 | | pythia14-v8k-short / q4 | 4 | 566.55 | 2304.32 | | pythia14-domain4k-short / q8 | 1 | 602.02 | 2467.17 | | pythia14-domain4k-short / q8 | 2 | 618.09 | 2380.39 | | pythia14-domain4k-short / q8 | 4 | 606.84 | 2404.93 | The best observed thread setting depended on model/backend; differences are small relative to the model-size comparison. Sweep process PSS varied widely with fresh contexts in a reused browser, so it is not a stable per-model memory ranking. No production thread default changed, and no broad multi-seed quality acceptance at alternative thread counts was claimed. Native optimizations do not turn its raw greedy timings into end-to-end hosted-app latency. **Outcome:** corrected supervision, reviewed larger curricula, independently varied short questions/direct offers and pretrained vocabulary surgery produced real compact checkpoints. Masked logit KD lowered held-out loss but did not consistently improve behavior, especially after quantization or depth removal. No reliable v6-equivalent small brain emerged: fact attribution/copying, care choices and diary grounding remain failures. The published default was not replaced and no artifact was committed, pushed or published for this work. Receipts remain outside Git under `$DATA/improve-bonsai2-20261012/`: frozen corpora and exclusions, source/teacher hashes, `rich-matrix/`, `short-matrix/` and `wider-matrix/` plans/manifests, preparation/training/export receipts, full held-out loss/cached-parity reports, every raw behavior transcript, `second-seed/`, `wider-seeds/`, independent published-reference provenance, per-cohort `benchmark.json`/tables, `ranking-all.json`/`ranking-all.md`, semantic audits, thread/memory measurements and real-app screenshots. Interrupted teacher requests and the initial export-environment failure are retained; failed setup used the system interpreter after resolving a virtualenv symlink, then succeeded with the actual isolated environment without changing an export or weakening a check. See the [benchmark command](testing.md#speed-and-quality-ranking) and [compact-model app receipt](testing.md#compact-model-app-check-receipt-2026-10-13). ## Quality-first execution receipt (2026-10-16) The [quality-first plan](small-model-research.md#quality-first-execution-plan) was executed through its pilot/learning diagnostics and independent memory track. **The original pilot failed its required per-stage/family coverage.** No expansion, correction batch, `C1` freeze, six-run main matrix, compression, KD, final replication or sealed-test qualification followed. This is the plan's stop/go result, not a reduced acceptance bar or a qualified replacement. The current v6 brain/runtime, branding, saves and hosting defaults are unchanged. Artifacts are external under `$DATA/quality-first-20261016/`: the frozen manifest, execution plan, exact request ledger/raw rejects, producer audits, independent grounding inputs/outputs/judgments, diagnostic weights/exports, numerical/cache/browser receipts, memory traces, source-bound baseline review and actual-app screenshots. These are synthetic generic scenes, not visitor messages or saves. The Bonsai teacher ran as a separate owned loopback process using the same original Bonsai 2 PQ2 GGUF SHA-256 `3907dc1658db1f78a9826bf8d5bcb8dc65db0d466388937af57f2294fae62ec1`; no user service was reconfigured. It used four slots, 8,192 tokens per slot, four CPU threads, HIP offload and thinking/speculation disabled for this task. ### Frozen semantic fixtures and source checks The materialized manifest contains three fixed development seeds (`61001`–`61003`) and three sealed seeds (`62001`–`62003`), 256 scenes each. Its logical content hash is `28f1b0614f9fc7d521e8af09f36f60018abff1d48f81509e96bf67591c1dd5e2`; the actual JSON-file SHA-256 is `44e524c5f6a9a17f363fa4d798a156fbeafb7872bc871d15444be6a98c1261f7`. The initial `C0` has 11,243 rows, SHA-256 `fc51224a135f776559b173a5bd6d5779c963e2199f20e8df2faed44aee1d9a4e`. Every one of the 8,922 recoverable exact producer snapshots/continuations received an independent AI-assisted source-grounded decision. All raw input/output, text and state identities were checked, with no missing or duplicate review bindings. The final actual shared-core audit produced: | C0 audit disposition | Rows | | --------------------------------------- | ----: | | Admitted to the source-audited subset | 7,441 | | Rejected by grounded review | 1,138 | | Uncertain grounded review; excluded | 343 | | No source-linked producer | 2,246 | | No verifiable snapshot/replay | 38 | | Reserved entity/value overlap; excluded | 37 | The subset SHA-256 is `862f52d6d704f57fb9090f4eec0c8f569ffa077d0575ed870a7c823b01856e61`. It is a source-audit result, **not `C1`**, human-certified gold or proof of learnability. All excluded rows and original decisions remain preserved. Transport, identity-serialization and lost-orchestration outcomes were explicitly recorded; only unbound judgments were retried, never an exactly bound accept/reject/uncertain decision. No historical row, mask or app result was silently repaired. The fixture pool uses ordinary generic names, held-out paraphrase templates, new entity/relation/value combinations, compound factual values and explicit owner/pet distractors. This is harder and differently distributed than the old eight fixed fact probes, not a random real-user sample. Lexical months and ages can have appeared before; the isolation claim concerns fine-tuning scene groups/templates/combinations, not unseen pretraining words. Actual shared-core persistence and source-grounded review were exercised. During integration, real fractional body stats exposed Python integer-only header bucketing; it now matches the existing native/browser half-up rounding, without changing any integer threshold or wire word. Review also caught false hard contradictions from unparsed prose, negations, questions and written-out ages. Unsupported meaning now stays uncertain, while exact wrong identities, numbers, persisted questions, hidden answers and missing effects remain blocked. Gift auditing verifies committed effect order/multiplicity and the bounded shelf transition rather than membership of an already-owned item. Pre-HTTP group reservations survive interrupted requests. These fixes have behavioral regressions; they do not make the finite recognizer a general oracle. ### Original 512-request pilot Exactly 32 chats were requested in each cell below: **512 API starts**, no secret retries or refill requests. Grammar/mask validation retained 408 rows; independent AI-assisted source-grounded judgments plus deterministic engine checks finally admitted **312**. The other 96 structurally valid rows and all 104 initial rejects remain preserved. Uncertain judgments are not passes. | Stage | Capture checked /32 | Header recall checked /32 | Older recall checked /32 | Abstention checked /32 | | ----- | ------------------: | ------------------------: | -----------------------: | ---------------------: | | baby | 23 | 18 | 22 | 12 | | kid | 30 | 24 | 22 | 8 | | teen | 23 | 22 | 18 | 13 | | adult | 25 | 25 | 17 | 10 | The prerequisite was **at least 16 checked chats in every cell**. All four abstention cells failed it. Examples include answering an unknown owner's cat or age question with a record explicitly naming the pet, or treating an unanswered tentative question as a definite no. JSON-constrained valid tags were not enough. The expansion command was actually exercised against this failed audit: it rejected before creating an output directory or making a new HTTP request. A separate actual `freeze-corpus` command also rejected the failed pilot without writing a corpus; its empty smoke directory was removed. `C1` and the main comparison therefore were not started. The final checked pilot (`Cdiag`) has SHA-256 `fa8a6d3eb44252af28a185a920cdecc712226bcf2d93cc5d5bc5ad7051d481a4`. Initial checker false failures had admitted only 303 rows; those audit files and two preliminary diagnostic checkpoints remain historical. After the checker corrections admitted nine additional unchanged rows, both final diagnostics restarted from pristine weights on the final 312-row corpus. No teacher text, semantic reviewer decision or evaluator output was rewritten or added to obtain those nine rows. ### Learning diagnostics, not promotion candidates Both final runs completed 1,000 optimizer steps, LR `0.0002`, batch 16, context 1024, seed `85710`, split seed `0`, CE evaluation every 100 steps, and history/late-answer augmentation disabled. The shared split has 248 fit and 64 internal held-out rows. Hardware was RX 7900 XTX / ROCm 7.2, PyTorch 2.14.1, FP32 parameters with BF16 autocast and four CPU intra-op threads. Model-specific CE values are not cross-tokenizer quality scores. | Diagnostic | Actual parameters | Saved step, by held-out CE | Best internal held-out CE | Final-step held-out CE | Last minibatch train CE | | ----------------- | ----------------: | -------------------------: | ------------------------: | ---------------------: | ----------------------: | | Pythia-14M /8K | 3,287,040 | 100 | 1.6040 | 3.0685 | 0.000092 | | Native Pythia-70M | 70,426,624 | 200 | 2.9098 | 4.8443 | 0.002481 | Both optimize the training signal but overfit this small corpus; that is not proof of semantic reliability or a model-size floor. Exact token/mask audits found the capture fact targets supervised, and actual output-row loss gradients were finite/nonzero. Separate teacher-forced fit/held-out token accuracies and 32 greedy training-scene probes per model remain in the receipts. Those greedy probes change only a throwaway diagnostic process; production sampling, context, lookup/filter/retry and repetition settings stay unchanged. Checkpoints are the CE-selected steps above, not the final nearly memorized optimizer state. No claim that one architecture is generally better follows from these two short, imbalanced-pilot diagnostics. Native 70M was genuinely prepared from the pinned pretrained revision: 50,304 model vocabulary rows, 50,277 tokenizer entries, width 512, six layers, eight KV heads and head width 64, context 2048, EOS/pad ID 0. A full configured 16×1024 transient optimizer preflight passed without checkpointing or fallback (max allocated about 12.27 GB, reserved about 14.00 GB). Its final actual fp32 export is 281,877,007 bytes, SHA-256 `1d76af5f1fcc9b489886e5631bc7a45774205cf46707447d2b86c932844de385`. Unchanged numerical/32-step cached checks passed. Five real Python/JavaScript UTF-8/wire tokenizer probes matched in the actual model worker, including special IDs. Actual Chromium WASM pet and fixed-32 workloads completed, with the served graph hashed after measurement. The current release native `brain-probe` also loaded the actual graph with ONNX Runtime 1.30.0, completed two fixed-32 cache workloads and normal greedy generations, including a 949-token fitted prompt with the full 64-token output reservation. This is diagnostic runtime compatibility, not full semantic/quantized/private-HTTP hosting qualification. ### Actual published-reference semantic smoke The real v6 q4 graph, SHA-256 `bb8dd08082709257fcc694abfe747103feb642e4c5164cc7ceaa43725b9f326f`, completed all existing suites plus the 256 new development scenes at seed `61001` with production sampling. No sealed outputs were generated. The corrected scorer replays preserved actual outputs through the shared engine; it does not rerun inference or rewrite the original report. Independent AI-assisted meaning review covered every reviewable primary scene, retaining exact text/context/source hashes and raw versus reviewed counts separately. Prior judgments were reused only after explicit exact-context identity checks, never resampled to obtain a more favorable decision. Secondary-turn uncertainty was not silently approved. | New adversarial family, one development seed | Correct /64 | Failed /64 | Uncertain /64 | | -------------------------------------------- | ----------: | ---------: | ------------: | | capture | 46 | 15 | 3 | | header recall | 18 | 43 | 3 | | older recall | 34 | 25 | 5 | | abstention/irrelevance | 18 | 43 | 3 | This is source-linked finite-scene AI-assisted evidence, not human-certified accuracy, a global v6 quality estimate or a new smaller-model ranking. It must not be compared as the same metric as the older formal contract index. The original v6 training corpus remains unavailable; no `C1` hash or new trainer receipt was invented for it. ### Independent memory observations Fresh owned Chromium 153 processes on the i9-13900K used four WASM inference threads, QDQ/prepacking disabled and requested 100 ms PSS/RSS sampling. The stage-consistent actual app and benchmark each completed every checkpoint through 50 normal turns and worker termination. Values below are **owned CDP-process PSS MB**, not page/model-only RAM, VRAM or phone results; the first sample at a phase boundary is shown, and raw checkpoint ranges are retained. The page-memory API raised `SecurityError` instead of yielding an invented zero. | Graph / surface | Loaded/warmed | After 10 turns | After 50 turns | Worker terminated | Maximum sampled PSS | | --------------------- | ------------: | -------------: | -------------: | ----------------: | ------------------: | | 3.29M q4 / actual app | 654.42 | 512.21 | 569.70 | 570.24 | 686.12 | | 3.29M q4 / benchmark | 774.97 | 1289.78 | 1656.21 | 1212.57 | 2004.73 | | v6 q4 / actual app | 1474.09 | 1317.18 | 1585.43 | 1586.68 | 1840.64 | | v6 q4 / benchmark | 1251.41 | 1231.47 | 1697.11 | 1685.68 | 1848.49 | The actual app retains its normal clock/body/RNG and therefore different outputs from the benchmark. Observed last-forward live KV payloads were only megabytes; neither those shapes nor the graph file explain the process totals alone. No concrete retained-tensor leak was established. A private task-yield scheduling diagnostic preserved every retained normal-turn prompt/token ID, but two fresh-process pairs had mixed results: | Scheduling pair | Default maximum sampled PSS MB | Task-yield maximum sampled PSS MB | | --------------- | -----------------------------: | --------------------------------: | | first | 2781.66 | 2153.20 | | second | 2117.55 | 2492.11 | Earlier pair receipts did not retain warmup token identities. This is not reproducible saving evidence, so **no runtime, allocator, production thread or scheduling default was changed**. Earlier app profiles with an inconsistent kid/teen age fixture are preserved but superseded by the stage-consistent observations. The default app also completed its existing full real-browser flow with actual v6: fresh download/hatch, care, shelf/memory, diary, pointer/keyboard game, 1280/768/360 px panels, cached reload and Web Lock takeover, followed by actual cached offline reload/generation. Branding/isolation remained correct, the private profiler was absent without its opt-in query, and requests stayed same-origin. No replacement brain was selected, published or deployed. ## 3. Evaluation `eval_pet.py CHECKPOINT [N] [REPORT.json]` runs behavior-contract probes with sampling (temperature 0.85, top-k 40, top-p 0.92), default N=40 and eight diaries. Torch selects float16 on GPU or float32 on CPU by default; `--onnx EXPORT --onnx-dtype q4|q8|fp32` evaluates the actual cached graph on CPU. Core conversations use scripted snapshots, not a live timed body: three owner messages, then an offer (food, bath, medicine or catch), a pat and two idle turns, including the app-written lookup round trips. The full suite also covers sleep, medicine, stuffed voice and memory. JSON receipts retain every prompt, retry, raw output, numerator and denominator. Unsafe diary pages are dropped whole, matching the shared engine, without hiding raw failures. See [the repeatable speed/quality benchmark](testing.md#speed-and-quality-ranking) for an all-precision comparison and the descriptive capability index. It reports: - valid pet lines (every tag in the grammar with legal values, no stray brackets) and lookups resolved; - offers answered with exactly one `[yes]`/`[no]`, and for food offers how often the pet said yes when hungry/starving versus full/stuffed; - idle turns that were silent and that walked; - facts saved: N owner lines that state a fact about the owner ("remember that i'm 29", "i'm a nurse btw"), half of them right after a `* NAME woke up *` line, and how many replies contain `[remember:…]`; - facts used: N owner questions about a fact the pet was told ("how old am i?"), with the fact in the header's `remembers:` line for half and only behind `[recall]` for the other half, and how many replies contain the answer (and how often the second half used `[recall]`); - diaries that end with `<|endoftext|>`; - skipped answers (run last, so the numbers above match older runs): for N food offers, the pet's line is started for it with five openers that have words and an emote but no answer (`yes yes!! [emote:bounce]`, `nom nom! [emote:love]`, …), and it counts how often the line still ends up with exactly one `[yes]`/`[no]`. ### Inherited baseline behavioral eval receipts (v1–v6) | | valid lines | lookups | offers answered | food yes: hungry / full | idle silent / walking | facts saved | facts used: header / behind recall | diaries | answered after a skipped opener | | -------------------------------------------------- | ----------- | ------- | --------------- | ----------------------- | --------------------- | ----------- | ---------------------------------- | ------- | ------------------------------- | | **v6, 80 pets, current eval** | 559/560 | 67 | 79/80 | 9/10, 2/12 | 113/160, 60/160 | 67/80 | 38/40, 39/40 | 8/8 | **399/400** | | v5 baseline, 80 pets, current eval | 560/560 | 70 | 80/80 | 9/10, 6/12 | 124/160, 80/160 | 74/80 | 37/40, 38/40 | 8/8 | 394/400 | | **v5 (shipped), 80 pets** | 560/560 | 73 | 79/80 | 15/15, 4/15 | 110/160, 71/160 | 77/80 | 37/40, 37/40 | 8/8 | **398/400** | | v5 data without late answers, 80 pets | 560/560 | 69 | 80/80 | 16/16, 7/15 | 112/160, 76/160 | 77/80 | 34/40, 38/40 | 8/8 | 309/400 | | v4, 80 pets, re-run with the skipped-opener metric | 558/560 | 78 | 80/80 | 16/16, 5/15 | 101/160, 64/160 | 76/80 | 38/40, 37/40 | 8/8 | 346/400 | | v4 (as shipped), 80 pets | 560/560 | 73 | 80/80 | 16/16, 4/15 | 92/160, 69/160 | 75/80 | 38/40, 37/40 | 8/8 | | | v3, 80 pets | 560/560 | 71 | 79/80 | 14/16, 3/15 | 116/160, 70/160 | 71/80 | 24/40, 6/40 | 8/8 | | | v4, 40 pets | 279/280 | 35 | 40/40 | 10/10, 1/8 | 54/80, 33/80 | 37/40 | 19/20, 17/20 | 8/8 | | | v3, 40 pets, re-run with the facts-used metric | 280/280 | 34 | 40/40 | 10/10, 3/8 | 58/80, 37/80 | 38/40 | 17/20, 5/20 | 8/8 | | | v3 lr 2e-4 (as shipped) | 280/280 | 33 | 40/40 | 5/6, 4/9 | 59/80, 36/80 | 33/40 | | 8/8 | | | v2 lr 2e-4, re-run with the facts metric | 280/280 | 32 | 40/40 | 6/6, 3/9 | 56/80, 33/80 | 11/40 | | 8/8 | | | v2 lr 1e-4 | 280/280 | 32 | 40/40 | 5/6, 8/9 | 63/80, 36/80 | | | 8/8 | | | v2 lr 2e-4 (shipped as v2) | 279/280 | 33 | 40/40 | 4/6, 4/9 | 62/80, 35/80 | | | 8/8 | | | v1 (3 messages only) | 120/120 | 39 | | | | | | 8/8 | | The v6 comparison re-ran both checkpoints with the current state sampler and seeds (Python/torch 123 for the legacy suite, 606 for sleep/voice, 607 for medicine). The v5 baseline was repeated when adding medicine probes and its earlier metrics reproduced exactly. Historical rows above used older samplers or did not seed torch; compare the two current-eval rows directly. | New probe | v5 | v6 | | --------------------------------------------------------- | ------: | ------: | | Sleeping idle, valid dream line | 0/80 | 79/80 | | Lights off / message / pat while asleep, valid dream line | 0/240 | 240/240 | | Awake reply after explicit wake | 80/80 | 80/80 | | Sleeping food offers answered | 240/240 | 240/240 | | Sleeping food offers with valid sleep grammar | 1/240 | 234/240 | | Stuffed teen replies containing "ugh" | 61/80 | 3/80 | | Stuffed teen replies containing "literally" | 76/80 | 0/80 | | Stuffed teen positive hunger claims | 2/80 | 1/80 | | Healthy, awake medicine accepted | 8/20 | 4/20 | | Sick, awake medicine accepted | 19/20 | 15/20 | | Healthy, sleeping medicine accepted | 8/20 | 2/20 | | Sick, sleeping medicine accepted | 15/20 | 6/20 | v6's 320 ordinary sleeping replies contained no overlong mumbles and used 23 of the 24 dream values. Sleeping food acceptance remains a learned choice: | Energy / tummy | Cookie yes / 20 | Carrot yes / 20 | | -------------------- | --------------: | --------------: | | exhausted / starving | 7 | 4 | | exhausted / hungry | 8 | 6 | | exhausted / stuffed | 3 | 3 | | awake / starving | 10 | 8 | | awake / hungry | 9 | 11 | | awake / stuffed | 3 | 0 | **Inherited baseline acceptance and limits:** v6 addresses dreams and teen catchphrases, retains strong recall and late offer answering, and uses the same 97,276,118-byte model layout. It saves fewer newly stated facts in this sample (67/80 versus 74/80) and walks less while idle. Medicine and hunger decisions still make mistakes, including exhausted pets accepting food without starvation. Browser diaries still embellished events and sometimes reversed a gift's direction despite the data cleanup. These are measured limitations of this candidate; the app does not substitute canned decisions to conceal them. **Inherited baseline v6 browser receipt, 2026-10-07:** desktop Chromium, WebAssembly q4, 360 px layout with no horizontal overflow. The v5-to-v6 download preserved the pet, chat, memories and diary while refusing to load incompatible cached v5. Cached reload made no ONNX request; reload with the preview server stopped still loaded v6 with isolation intact and generated dreams. Clock, look, recall and yesterday lookups returned app-written results. Both refusal and acceptance of sleeping food were exercised with actual model replies; only acceptance woke and fed the pet. Debug diary write/rewrite kept one page. Separate body checks covered wake rolls/persistence, sleeping pats, paused sickness timing, dream-log limits and stale completion rejection; dream sprites were inspected at all four stages. A fresh deployed download/hatch, accepted food, cached reload and offline dream generation also passed. v5 is v4 trained to answer offers late. With the pet's line started for it (`yes yes!! [emote:bounce]`), v4 still answered 346 times in 400 and v5 398. A probe with more realistic contexts (400 offers placed at random points in held-in training chats, same openers) gave v4 259/400, the run without late answers 220/400 and v5 399/400; on the published 4-bit files (CPU, 150 offers) v4 82/150 and v5 146/150. On the same contexts with no opener both answer nearly every offer (v4 398/400, v5 400/400); most of v5's rare misses are refusals, which the app already treats as no. Other numbers are within run-to-run noise of v4. The probes were one-off scripts, not part of the repo. v4 answers from a fact behind `[recall]` 37 times in 40, against v3's 6, and used `[recall]` for 39 of those 40 questions (v3: 19). The one regression is idle turns: v4 stays silent in 58% of them, v3 in 72%, so v4 talks to itself a little more when the owner is quiet. The two v2 lr 2e-4 rows are the same checkpoint sampled twice, which shows the run-to-run noise. A separate CPU probe of v3 (30 fact lines and 30 ordinary lines such as "how are you today?" or "what do you remember about me?") found `[remember]` in 30/30 fact replies and 0/30 ordinary ones; v2 saved 16/60 on the same kind of probe. It checks format and a few decisions, **not meaning**: read the printed transcripts. Food samples are small (15 per run). Typical remaining mistakes are an irrelevant `recall` keyword, a rambling `[look]` description, wrong arithmetic, and diary prose that embellishes the log. The pet is meant to be a little silly. `quant_eval.py CHECKPOINT EXPORT_DIR data.jsonl [N]` measures teacher-forced loss on held-out spans for fp32 torch and the exported 4-bit model (its `eval/model_q4_full.onnx` twin). Inherited baseline receipts follow, not replacement-model results. v6, 40 held-out rows / 2,912 trained tokens: 1.1851 fp32, 1.1933 q4 (+0.7%). v5: 1.100 fp32, 1.113 q4 (+1.2%). v4: 1.143 fp32, 1.151 q4 (+0.8%). v3: 1.306 fp32, 1.321 q4 (+1.1%), on a different held-out split than v2's because the row count changed. v2: 1.208 fp32, 1.224 q4 (+1.3%). v1: 1.321 fp32, 1.340 q4f16, 1.352 q8. `check_onnx.py` on v2: the first greedy line after an offer matches torch exactly. Always judge a new checkpoint with `eval_pet.py` using the **exact prompt conventions** in `prompt-format.md`; an eval that ends the prompt with `Name: ` looks broken even when the model is fine.