Data, training and evaluation #

Scripts: train/gen_data.py, train/mix_data.py, train/train.py, train/eval_pet.py, train/quant_eval.py, train/teachers.sh, train/migrate_v6.py. Everything here runs in one Python environment (torch 2.14 + CUDA, transformers 5.19; $VENV below). Datasets, checkpoints, exports and logs live in a data directory outside the repository ($DATA below).

1. Teacher data #

gen_data.py OUT.jsonl N_CHATS N_DIARIES asks a large model to write one chat log or diary entry per request and keeps only samples that pass validation.

State sampling. rand_state() builds a random pet: stage (baby 2 : kid, teen, adult 3 each), two traits, random time, weather, stats, up to six memories drawn from sentence templates, a six-entry day log, shelf, toy, poop count, asleep and sick flags and a "yesterday" line. The header it renders is the exact production header (world.header). Since v2, "pooped" is kept in only 40% of day-log draws and the room has poop in 2 of 7 states (v1: 2 of 5), which reduces the poop skew described below.

Prompt. A long static block states the rules (tag grammar, allowed values, no gender assumptions, one line per turn) and ends with a task block holding the names, the header, allowed event lines, length, personality, stage voice, the true lookup results and one to three scenario ideas from a list of 48 (greetings, asking the time, recall questions, gifts, comfort, boundaries, dancing, trivia, goodnight, offers, pats, wandering, idling and so on). Twelve of them exercise the pet's own decisions (ACTION_SCENARIOS), and 80% of chats include at least one. The rules explain offers (answer with exactly one [yes]/[no], decided from the needs line and personality), pat reactions, walking and silent lines (at most two in a row), and ban stage directions. Since v3 they also state the memory rule: when the owner says something about themselves, and always on "remember that …", the pet saves it with [remember:FACT] in the same reply (even sleepy or just woken), while [recall:…] only looks memories up. --focus memory puts one of seven MEMORY_SCENARIOS into every chat; it was used for the v3 top-up. Since v4 the rules also say how to answer from memory: facts are saved with the words someone would ask about ("Sam is 29 years old"); the pet answers from the remembers: line when the fact is there, and otherwise looks it up with [recall:WORD] (one word that appears in the saved fact, because recall matches whole words) and answers from the result in the same line, or says it does not remember when the result is {nothing}. --focus recall builds every chat around one question about a fact from RECALL_FACTS (fifteen kinds: age, pets' and people's names, job, birthday and so on), placed older than the header's last three memories (55%, so only [recall] finds it), in the header (25%), or never told (20%). Other memories that share a word with the fact are left out so the lookup's result is the fact alone. check_recall rejects a row whose pet does not say the fact (or, for a fact never told, says it before being told); for older facts it also requires a [recall] that found something. Keeping the static block first lets the teacher's prefix cache serve it, which roughly doubled throughput.

v6 sleep and stage focus. --stage STAGE forces the stage for both chats and diaries. --focus sleep samples mostly sleeping states with energy 0–95, plus awake-to-nap cases at energy 7–35. Talk, pat and lights-on outcomes use world.wake_chance, identical to the browser formula. The generator gives the teacher an ordered list of DREAM, AWAKE, NAP or sleeping OFFER turns and asks for a JSON array of pet lines. It assembles the app events itself and inserts woke up immediately after an accepted sleeping offer. The teacher still chooses each dream, mumble, awake reply and offer decision. Unfocused chats that start asleep also use this path. No dream text is canned.

The teen voice is sassy but sincere, with varied wording instead of mandated catchphrases. Memory fillers avoid duplicating any starting-memory template, preventing conflicting ages in recall scenarios. Diary signatures use the actual pet name. Diary guidance explicitly identifies the pet as the subject of unlabelled log entries, keeps gift direction and owner speech intact, and does not turn remembered people or activities into events from today. Teen entries use 3–5 short sentences, with shorter logs kept brief. Dream entries are sampled in day logs; owner quotes are excluded from chat headers but kept in diary headers.

Validation (check_chat, check_diary). A sample is rejected unless:

• every line is an owner line, a pet line or an allowed event line;
• owner and event lines contain no tags, braces or brackets;
• owner lines do not narrate events: no event line in the owner's mouth (Sam: * Sam patted Mochi *) and no care narration (Sam: fed Mochi a rice ball, gave you a bath). Only the app writes events;
• a played catch with … (N stars) result follows a [yes] to a catch offer;
• every tag is in the grammar with an allowed value; emote, want, gift and walk values come from the fixed lists (a small alias table fixes common slips like [emote:surprised] → wow);
• an offer event is answered by the pet's next line with exactly one [yes]/[no], and [yes]/[no] appear nowhere else;
• no more than two silent pet lines in a row, at most one [walk:…] per line, and no asterisks left after short *stage directions* are stripped;
• lookup results are replaced with the true values from the state, and a recall the teacher claimed was empty or non-empty must agree with world.recall; recall searches the starting memories plus every [remember:…] earlier in the same chat, as the app does (through v2 it saw only the starting memories, so "remember X … recall X" taught {nothing});
• pet lines are under 220 characters with no stray brackets, and there are at least two of them;
• a sleep state machine enforces one dream with at most four mumbled words; an offer refusal adds [no], while a sleeping [yes] has no other tags and requires woke up immediately after. Exhausted sleeping pets accept food only when starving. Awake dreams, repeated sleep, wakes directly before offers or after a message/pat, and spontaneous instant wakes after naps are rejected;
• memory tags name the owner; memory/note tags have no gendered pronouns; recall queries are at most two words and owner lines contain letters or digits;
• narrow positive first-person hunger claims while full/stuffed are rejected; accepting medicine without sickness in the header is also rejected; fuzzier invented history and mood issues are handled by teacher guidance and transcript review, not claimed to be fully detectable by regex;
• headers cannot contain conflicting remembered ages or chat OWNER said: entries, and chat weather must belong to the hour's day/night list;
• diaries start dear diary,, use only valid [doodle:…] tags, and contain no remaining brackets or doodle: fragments or literal - PET signature.

v1 rejected about 22% of samples, mostly for invented tag values and mismatched recalls. v2 rejected about 21%, mostly for [yes]/[no] outside an offer, then owner lines with tags and invented emote values.

Re-checking stored rows (recheck). gen_data.recheck(row) applies the same line rules to a stored row, keeping the lookup results already in it. It also requires canonical text and a valid loss mask: span endpoints must be integers within the text, and their combined coverage must match the pet words/tags or diary body rebuilt by the parser. Equivalent merged, split, reordered or overlapping intervals are accepted; missing, shifted or incorrect coverage is rejected with ValueError, not silently repaired. Owner/event lines, headers, pet-name prefixes and lookup results remain untrained. train.py re-checks every row before training and refuses to start if any row fails, listing the reasons; --drop-stale drops those rows instead. mix_data.py and quant_eval.py apply the same check, so data written for an older app cannot quietly teach behavior the app no longer has. When a rule or the app's behavior changes, add the rule to check_body and the next run reports every older row it catches.

v6 migration. python3 train/migrate_v6.py OLD.jsonl OUT.jsonl NEW.jsonl … removes all old teen rows and filters visible chat-header quotes. It first rebuilds valid chat bodies without inventing events, preserving failed wake rolls and their dream replies. For legacy bodies needing a wake, it inserts a possible wake before a message/pat or after lights on only when an awake reply requires one; earlier no-wake dreams in mixed transcripts are retained. It also wakes after an asleep [yes]. Chat and diary text and loss spans are rebuilt with the current parser before re-checking, repairing their masks only in this explicit migration path and rejecting remaining stale rows. It does not fabricate replacement dreams. Old headers store only six log entries, so earlier filtered-out history cannot be recovered. Every new row is also checked; counts are printed by source, kind and first rejection reason. Optional --exclude FILE reads a JSON object mapping SHA-256 of row text to a confirmed manual-review reason; it matches before or after migration. The script uses arguments only, with no environment variables. test/data_rules.py checks it against a stand-in.

History augmentation chooses donors whose final sleep state matches the next row's initial state, including both boundaries when it prepends two donors. Donor text remains untrained. This prevents stale donor naps from contradicting the trained awake continuation.

Spans. Each row stores text plus spans, the character ranges that carry loss: the pet's words and tags, and the diary body after the fixed dear diary, prefix. Owner lines, event lines, headers, pet-name prefixes and {results} are excluded.

Running the teacher #

gen_data.py needs any OpenAI-compatible chat endpoint (--endpoints, --model, or TEACHER_ENDPOINTS and TEACHER_MODEL; there are no defaults), but a shared model server is sized for a few concurrent users and is far too slow for bulk generation. For tokenling-v1 we ran dedicated batch servers, two independent vLLM instances of a W4A16-quantized 27B-class instruction model, one per GPU (24 concurrent sequences, 4,096-token context, prefix caching on), on ports 19400 and 19401. Each card holds the whole model plus a 60K-token KV pool, so the copies need no per-layer communication and fail independently. The generator sends each request to the less loaded instance. With about 50 workers the pair yielded 2.3 accepted samples per second (roughly 115–135 generated tokens/s per instance under load). tokenling-v2 used the same setup: about 2.2 requests/s, 1.7 accepted chats/s with the stricter v2 validator.

train/teachers.sh start | wait | stop is the generic version of that setup: one vllm serve per GPU with the same defaults, configured through environment variables (TEACHER_MODEL is required; GPUS, PORT_BASE, MAX_SEQS, MAX_LEN, GPU_UTIL, VLLM, VLLM_ARGS and LOGS are optional; the script header lists them). Put anything your model or vLLM install needs, such as quantization or attention-backend flags, in VLLM_ARGS. Honest status: the tokenling-v1 data was generated with a machine-specific launcher for the quantized 27B model. The generic script was tested end to end (start, health wait, a completion, stop, GPU memory released) with a small stand-in model, not re-run at full scale with the original teacher.

Two instances were chosen over tensor parallelism by reasoning, not by benchmark. If your GPUs are linked by NVLink, tensor parallelism is cheaper than over PCIe and may be competitive. Measure --tensor-parallel-size 2 with a larger sequence cap before assuming two instances are best for a bigger run. Sharing GPUs with other services is covered in deployment.md.

train/teachers.sh start && train/teachers.sh wait
python3 train/gen_data.py $DATA/v2/new.jsonl 9000 0 --workers 52 --seed 21 \
  --endpoints http://127.0.0.1:19400,http://127.0.0.1:19401 --model TEACHER_NAME
train/teachers.sh stop
python3 train/mix_data.py $DATA/data.jsonl $DATA/v2/new.jsonl $DATA/v2/data.jsonl

For v2 the generator was stopped after 6,600 of the 9,000 requests (5,221 accepted chats), which was enough, to give the GPUs back sooner. Diaries were not regenerated: the diary format did not change.

For the v3 memory top-up, with the same two-instance setup:

python3 train/gen_data.py $DATA/v3/memory.jsonl 2200 0 --focus memory --workers 52 --seed 31 \
  --endpoints http://127.0.0.1:19400,http://127.0.0.1:19401 --model TEACHER_NAME
cat $DATA/v2/data.jsonl $DATA/v3/memory.jsonl > $DATA/v3/data.jsonl

All 2,200 requests ran in about 14 minutes; 1,804 were accepted (18% rejected, mostly [yes]/[no] outside an offer, recall mismatches, malformed lines and invented emote values). The v2 rows are already in the current format, so they are concatenated rather than passed through mix_data.py, which would drop them for their newer tags.

For the v4 recall top-up, with the same setup:

python3 train/gen_data.py $DATA/v4/recall.jsonl 3000 0 --focus recall --workers 52 --seed 41 \
  --endpoints http://127.0.0.1:19400,http://127.0.0.1:19401 --model TEACHER_NAME
python3 train/gen_data.py $DATA/v4/memory.jsonl 700 0 --focus memory --workers 52 --seed 42 \
  --endpoints http://127.0.0.1:19400,http://127.0.0.1:19401 --model TEACHER_NAME
cat $DATA/v3/data.jsonl $DATA/v4/recall.jsonl $DATA/v4/memory.jsonl > $DATA/v4/data.jsonl

The 3,000 recall requests took 21 minutes and 2,308 were accepted (23% rejected: 196 [recall] rule failures and 45 recall mismatches from check_recall, 118 [yes]/[no] outside an offer, invented emote values, malformed lines). The 700 memory requests took 5 minutes, 581 accepted (17% rejected).

Mixing in older data (mix_data.py) #

mix_data.py OLD NEW OUT re-checks every row of both files with recheck and keeps the ones that pass, and also drops old chats that use [walk] (v1 had no walking). For v2 it kept 2,031 of 4,547 v1 chats (2,333 dropped for old care events, 183 for catch without an offer) and all 1,458 diaries.

Through v4 it only checked lines that start with * , so retired care events that a teacher had written as owner speech (Bea: * Bea fed Byte a rice ball *, mo: fed Momo a rice ball) got through, followed by the pet eating happily with no [yes]. See the v5 data below.

What is in tokenling-v6's data #

13,409 rows, generated and reviewed on 2026-10-07. Every row passes the current rules; a separate full audit rebuilt the exact loss spans, checked that all 5,924 lookup results are masked, and found zero duplicate texts.

Source	Requests	Initially accepted	Retained
Migrated v5 chats and diaries	—	12,727	8,517
Sleep chats, seed 611	2,500	1,961	1,836
Teen general chats, seed 612	2,100	1,334	1,265
Teen memory chats, seed 613	1,000	680	665
Teen recall chats, seed 614	1,000	676	669
Replacement teen diaries, seed 616	600	503	457

The first teen diary batch (550 requests, seed 615, 399 accepted) was superseded entirely after review found actor reversals, invented care events and repetitive "dramatic" phrasing. The replacement used the more explicit diary guidance above. Three Luna reviewers checked samples of old, sleep, general, memory and recall data, repeated-line candidates, and all 503 replacement diaries. Findings were verified against the original rows; false flags were retained. Confirmed manual exclusions total 69 rows: five old, one sleep, ten general, six memory, one recall and 46 replacement diaries. This is semantic review coverage, not a guarantee of perfect prose.

Migration replaces all 3,436 old teen rows. Other old-row drops, counted by the first failing rule, are: instant wake after sleep 198, sleep dream grammar 194, medicine while healthy 138, wake while awake 66, stray diary brackets 48, memory without owner 35, long recall query 26, wake after message/pat 22, gendered memory/note 20, owner without words 10, hunger while full five, literal PET signature five, manual review five, conflicting ages one, and sleeping yes with other tags one. Of the new chats, 113 sleep rows had daytime weather at night (a generator bug fixed before further generation), 11 accepted food while exhausted without starvation, 65 accepted medicine while healthy, and nine woke after a message/pat. Manual exclusions account for the remaining difference between initially accepted and retained rows.

Stage	Chats	Diaries
baby	2,226	217
kid	3,402	379
teen	3,078	457
adult	3,241	409

There are 4,076 sleeping pet lines and 3,944 dream tags, covering all 24 dream values. Other trained chat tags include 4,020 remembers, 3,256 recalls, 5,794 yes, 2,090 no and 7,036 walks. In 13,316 teen pet lines, "ugh" occurs in 328 (2.46%) and "literally" in 37 (0.28%), down from approximately 23% and 26% in the old teen data. The final dataset SHA-256 is aa2a838620cc193cfc942dd8447be889f12f63cc0fcf3c095d4941a958d1f57d.

What is in tokenling-v5's data #

12,727 rows: v4's 13,403 rows re-checked with the v5 rules (mix_data.py /dev/null v4.jsonl v5.jsonl). Dropped 676 chats: 377 with an event line in the owner's mouth, 165 with care narrated by the owner, 110 with a catch result after no catch offer, 23 with a fake event left in a pet line and 1 over-long pet line. No new teacher data.

What is in tokenling-v4's data #

13,403 rows: the 10,514 v3 rows below, 2,308 recall-focused chats (baby 425, kid 632, teen 626, adult 625) and 581 more memory-focused chats (101 start with the pet waking up). Trained tool calls in the recall chats: emote 8.3K, recall 1.9K, walk 1.6K, yes 1.4K, want 870, remember 770, sleep 460, diary 310, no 250, look 160, gift 160, clock 64, yesterday 59; in the memory chats: emote 2.2K, remember 650, walk 430, yes 330, want 190, recall 160 and fewer of the rest. That is about six times v3's 370 [recall] calls in the newest rows. Why: v3 saved facts but rarely used them. Asked about a fact that was only behind [recall], it answered correctly 5 times in 20 and often made one up ("how old am i?" → "five!").

What is in tokenling-v3's data #

10,514 rows: the 8,710 v2 rows below plus 1,804 new memory-focused chats (baby 330, kid 501, teen 494, adult 479; 360 start with the pet waking up). Trained tool calls in the new chats: emote 7.3K, remember 2.1K, walk 1.3K, yes 1.1K, want 630, recall 370, sleep 310, look 270, no 260, diary 220, gift 170, yesterday 95, clock 82. That roughly triples the dataset's [remember] calls (1.0K in v2). Why: v2 saved only about a quarter of the facts owners told it, and answered "remember that …" with [recall] or [clock]; in the v2 data only 5 of 60 explicit "remember that" owner lines were answered with [remember].

What is in tokenling-v2's data #

8,710 rows: 7,252 chats (5,221 new, 2,031 from v1) and 1,458 diaries (v1), across all four stages (baby 1,349/246 chats/diaries, kid 2,036/396, teen 1,960/399, adult 1,907/417). Trained tool calls in chats: emote 29.3K, walk 4.9K, want 4.0K, yes 3.7K, no 1.2K, diary 1.8K, sleep 1.6K, gift 1.5K, look 1.5K, recall 1.2K, remember 1.0K, clock 680, yesterday 570; 3.2K doodles in diaries. About 21% of new pet lines are silent. Offers are answered yes about three times out of four.

Known skews. Poop: v1 had poop in about a third of rows, mostly through the today: line and [look] results; v2's sampler lowers both for the new rows, but the reused v1 rows keep their share. The teacher is generous with food: pets often accept snacks even when full. recall keywords were chosen by the teacher and are sometimes irrelevant, and the dataset has no multi-day continuity.

What was in tokenling-v1's data #

6,005 rows: 4,547 chats and 1,458 diaries, for GPT-2 Medium, with direct care events, no offers, walking or silent lines.

2. Fine-tuning #

cd $DATA/v2     # holds data.jsonl; ../base/gpt2 is the downloaded checkpoint
CUDA_VISIBLE_DEVICES=0 $VENV/bin/python REPO/train/train.py \
  --data data.jsonl --base ../base/gpt2 --out ckpt-lr2e-4 --lr 2e-4

• Full fine-tune of all 124M parameters of GPT-2 (small): fp32 weights with bf16 autocast, gradient checkpointing, AdamW (β 0.9/0.95, weight decay 0.05), cosine decay to 10% with 20-step-minimum warmup, gradient clip 1.0, batch 16, 3 epochs.
• Loss masking. A token has a label only if its character range overlaps a stored span. Diary rows also get an <|endoftext|> label so entries end.
• Late answers. For a share of offer answers (--late-answer 0.3, about 2,300 lines in v5) the [yes]/[no] moves to the end of the line, after the words and emotes; the line keeps its length, so the spans still fit. Teachers answer first in 99% of lines, so v4, once it had sampled an emote before answering, usually treated the answer as skipped and the food was never eaten. The app applies tags in any order. Uses its own random generator, so the held-out split and history draws are unchanged.
• Stale rows. Every row is re-checked first (recheck, above); the run stops if any fail, unless --drop-stale.
• History augmentation. A quarter of chat rows (--history-aug 0.25) get one or two other chats' lines prepended with names swapped and no loss, so the pet stays in character when the context holds earlier conversation.
• Held-out set. The first max(64, N/50) rows of a seeded shuffle (174 for 8,710 rows, 210 for 10,514). quant_eval.py reuses the same seed to pick rows from that set, so keep the shuffle's random state untouched before it.
• The best-eval checkpoint is saved, evaluated every 200 steps.

Receipt, tokenling-v6 (2026-10-07): full fine-tune from the original GPT-2 607a30d base, lr 2e-4, three epochs, batch 16, seed 0, --history-aug 0.25 with matching sleep boundaries, and --late-answer 0.3 (2,101 answers moved). The 13,409-row dataset yields 16,002 training sequences after augmentation: 3,922,490 tokens, 1,174,065 carrying loss. Held-out loss on 268 rows fell from 3.4832 to 1.1524; the best checkpoint was the final step, 3,001. Training took 17 minutes 28 seconds on one RTX 3090, with four OpenMP CPU threads; the training loop averaged about 0.34 seconds per step including held-out checks. Phone performance remains unmeasured.

Receipt, tokenling-v5 (2026-10-07): same base and settings as v4, lr 2e-4, plus --late-answer 0.3 (2,261 answers moved in the training split), on 12,727 rows (v4's data with 676 stale rows dropped). 15,212 training sequences (4.02M tokens, 1.26M with loss), 2,853 steps at about 0.36 s/step on one RTX 3090, about 18 minutes. Held-out loss (254 rows) 3.45 → 1.115. A first run on the same rows without late answers (also 1.114) did not fix the offers (evaluation, below) and was not shipped. One GPU was used from 11:01 to 11:50 for both runs and all evals; nothing had to be stopped for it.

Receipt, tokenling-v4 (2026-10-07): same base and settings as v3, lr 2e-4, on 13,403 rows (v3's data plus the recall and memory top-ups). 16,051 training sequences (4.28M tokens, 1.34M with loss), 3,010 steps at about 0.37 s/step on one RTX 3090, 19 minutes. Held-out loss (268 rows) 3.49 → 1.165. GPUs were held from 03:37 to 04:28, data generation, training and all evals included.

Receipt, tokenling-v3 (2026-10-06): same base and settings as v2, lr 2e-4 only, on 10,514 rows (v2's data plus the memory top-up). 12,516 training sequences (3.31M tokens, 1.07M with loss), 2,347 steps at about 0.36 s/step on one RTX 3090, 14 minutes. Held-out loss (210 rows) 3.48 → 1.209. GPUs were held from 17:52 to 18:28 PDT, data generation, training and both evals included.

Receipt, tokenling-v2 (2026-10-06): base openai-community/gpt2 revision 607a30d. 10,356 training sequences (2.71M tokens, 880K with loss), 1,942 steps. Two runs in parallel, one per RTX 3090: lr 1e-4 (about 0.36 s/step, 12 minutes) and lr 2e-4 (about 0.75 s/step, 25 minutes, on a card that was running at reduced clocks). Held-out loss fell from 3.50 (base model) to 1.318 (1e-4) and 1.304 (2e-4, shipped); 2e-4 was lower at every matched step and its food answers followed the tummy better (below). Peak GPU memory about 13 GB. The allocator logs OOM retries on the longest batches, which is harmless. GPUs were held from 16:06 to 17:25 PDT, data generation included.

Receipt, tokenling-v1: GPT-2 Medium (355M), 7,018 training sequences (2.03M tokens, 717K with loss), 1,316 steps at about 1.1 s/step, about 25 minutes, held-out loss 3.20 → 1.313 at lr 6e-5.

3. Evaluation #

eval_pet.py CHECKPOINT [N] runs the app's real chat loop with sampling (temperature 0.85, top-k 40, fp16) on N random awake pets: three owner messages, then an offer (food, bath, medicine or catch), a pat and two idle turns, including the lookup round trips; then it writes eight diaries. It reports:

• valid pet lines (every tag in the grammar with legal values, no stray brackets) and lookups resolved;
• offers answered with exactly one [yes]/[no], and for food offers how often the pet said yes when hungry/starving versus full/stuffed;
• idle turns that were silent and that walked;
• facts saved: N owner lines that state a fact about the owner ("remember that i'm 29", "i'm a nurse btw"), half of them right after a * NAME woke up * line, and how many replies contain [remember:…];
• facts used: N owner questions about a fact the pet was told ("how old am i?"), with the fact in the header's remembers: line for half and only behind [recall] for the other half, and how many replies contain the answer (and how often the second half used [recall]);
• diaries that end with <|endoftext|>;
• skipped answers (run last, so the numbers above match older runs): for N food offers, the pet's line is started for it with five openers that have words and an emote but no answer (yes yes!! [emote:bounce], nom nom! [emote:love], …), and it counts how often the line still ends up with exactly one [yes]/[no].

	valid lines	lookups	offers answered	food yes: hungry / full	idle silent / walking	facts saved	facts used: header / behind recall	diaries	answered after a skipped opener
v6, 80 pets, current eval	559/560	67	79/80	9/10, 2/12	113/160, 60/160	67/80	38/40, 39/40	8/8	399/400
v5 baseline, 80 pets, current eval	560/560	70	80/80	9/10, 6/12	124/160, 80/160	74/80	37/40, 38/40	8/8	394/400
v5 (shipped), 80 pets	560/560	73	79/80	15/15, 4/15	110/160, 71/160	77/80	37/40, 37/40	8/8	398/400
v5 data without late answers, 80 pets	560/560	69	80/80	16/16, 7/15	112/160, 76/160	77/80	34/40, 38/40	8/8	309/400
v4, 80 pets, re-run with the skipped-opener metric	558/560	78	80/80	16/16, 5/15	101/160, 64/160	76/80	38/40, 37/40	8/8	346/400
v4 (as shipped), 80 pets	560/560	73	80/80	16/16, 4/15	92/160, 69/160	75/80	38/40, 37/40	8/8
v3, 80 pets	560/560	71	79/80	14/16, 3/15	116/160, 70/160	71/80	24/40, 6/40	8/8
v4, 40 pets	279/280	35	40/40	10/10, 1/8	54/80, 33/80	37/40	19/20, 17/20	8/8
v3, 40 pets, re-run with the facts-used metric	280/280	34	40/40	10/10, 3/8	58/80, 37/80	38/40	17/20, 5/20	8/8
v3 lr 2e-4 (as shipped)	280/280	33	40/40	5/6, 4/9	59/80, 36/80	33/40		8/8
v2 lr 2e-4, re-run with the facts metric	280/280	32	40/40	6/6, 3/9	56/80, 33/80	11/40		8/8
v2 lr 1e-4	280/280	32	40/40	5/6, 8/9	63/80, 36/80			8/8
v2 lr 2e-4 (shipped as v2)	279/280	33	40/40	4/6, 4/9	62/80, 35/80			8/8
v1 (3 messages only)	120/120	39						8/8

The v6 comparison re-ran both checkpoints with the current state sampler and seeds (Python/torch 123 for the legacy suite, 606 for sleep/voice, 607 for medicine). The v5 baseline was repeated when adding medicine probes and its earlier metrics reproduced exactly. Historical rows above used older samplers or did not seed torch; compare the two current-eval rows directly.

New probe	v5	v6
Sleeping idle, valid dream line	0/80	79/80
Lights off / message / pat while asleep, valid dream line	0/240	240/240
Awake reply after explicit wake	80/80	80/80
Sleeping food offers answered	240/240	240/240
Sleeping food offers with valid sleep grammar	1/240	234/240
Stuffed teen replies containing "ugh"	61/80	3/80
Stuffed teen replies containing "literally"	76/80	0/80
Stuffed teen positive hunger claims	2/80	1/80
Healthy, awake medicine accepted	8/20	4/20
Sick, awake medicine accepted	19/20	15/20
Healthy, sleeping medicine accepted	8/20	2/20
Sick, sleeping medicine accepted	15/20	6/20

v6's 320 ordinary sleeping replies contained no overlong mumbles and used 23 of the 24 dream values. Sleeping food acceptance remains a learned choice:

Energy / tummy	Cookie yes / 20	Carrot yes / 20
exhausted / starving	7	4
exhausted / hungry	8	6
exhausted / stuffed	3	3
awake / starving	10	8
awake / hungry	9	11
awake / stuffed	3	0

Acceptance and limits: v6 addresses dreams and teen catchphrases, retains strong recall and late offer answering, and uses the same 97,276,118-byte model layout. It saves fewer newly stated facts in this sample (67/80 versus 74/80) and walks less while idle. Medicine and hunger decisions still make mistakes, including exhausted pets accepting food without starvation. Browser diaries still embellished events and sometimes reversed a gift's direction despite the data cleanup. These are measured limitations of this candidate; the app does not substitute canned decisions to conceal them.

Browser receipt, 2026-10-07: desktop Chromium, WebAssembly q4, 360 px layout with no horizontal overflow. The v5-to-v6 download preserved the pet, chat, memories and diary while refusing to load incompatible cached v5. Cached reload made no ONNX request; reload with the preview server stopped still loaded v6 with isolation intact and generated dreams. Clock, look, recall and yesterday lookups returned app-written results. Both refusal and acceptance of sleeping food were exercised with actual model replies; only acceptance woke and fed the pet. Debug diary write/rewrite kept one page. Separate body checks covered wake rolls/persistence, sleeping pats, paused sickness timing, dream-log limits and stale completion rejection; dream sprites were inspected at all four stages. A fresh deployed download/hatch, accepted food, cached reload and offline dream generation also passed.

v5 is v4 trained to answer offers late. With the pet's line started for it (yes yes!! [emote:bounce]), v4 still answered 346 times in 400 and v5 398. A probe with more realistic contexts (400 offers placed at random points in held-in training chats, same openers) gave v4 259/400, the run without late answers 220/400 and v5 399/400; on the published 4-bit files (CPU, 150 offers) v4 82/150 and v5 146/150. On the same contexts with no opener both answer nearly every offer (v4 398/400, v5 400/400); most of v5's rare misses are refusals, which the app already treats as no. Other numbers are within run-to-run noise of v4. The probes were one-off scripts, not part of the repo.

v4 answers from a fact behind [recall] 37 times in 40, against v3's 6, and used [recall] for 39 of those 40 questions (v3: 19). The one regression is idle turns: v4 stays silent in 58% of them, v3 in 72%, so v4 talks to itself a little more when the owner is quiet.

The two v2 lr 2e-4 rows are the same checkpoint sampled twice, which shows the run-to-run noise. A separate CPU probe of v3 (30 fact lines and 30 ordinary lines such as "how are you today?" or "what do you remember about me?") found [remember] in 30/30 fact replies and 0/30 ordinary ones; v2 saved 16/60 on the same kind of probe.

It checks format and a few decisions, not meaning: read the printed transcripts. Food samples are small (15 per run). Typical remaining mistakes are an irrelevant recall keyword, a rambling [look] description, wrong arithmetic, and diary prose that embellishes the log. The pet is meant to be a little silly.

quant_eval.py CHECKPOINT EXPORT_DIR data.jsonl [N] measures teacher-forced loss on held-out spans for fp32 torch and the exported 4-bit model (its eval/model_q4_full.onnx twin). v6, 40 held-out rows / 2,912 trained tokens: 1.1851 fp32, 1.1933 q4 (+0.7%). v5: 1.100 fp32, 1.113 q4 (+1.2%). v4: 1.143 fp32, 1.151 q4 (+0.8%). v3: 1.306 fp32, 1.321 q4 (+1.1%), on a different held-out split than v2's because the row count changed. v2: 1.208 fp32, 1.224 q4 (+1.3%). v1: 1.321 fp32, 1.340 q4f16, 1.352 q8. check_onnx.py on v2: the first greedy line after an offer matches torch exactly.

Always judge a new checkpoint with eval_pet.py using the exact prompt conventions in prompt-format.md; an eval that ends the prompt with Name: looks broken even when the model is fine.