sample
+Generate candidate tokens from a base model or your current adapter.
+WORKING OBJECTS
+ARTIFACTS
++ Things built to think with. Notes become instruments here: small, + specific, and usable before they are finished. +
+WORKING OBJECTS
++ Things built to think with. Notes become instruments here: small, + specific, and usable before they are finished. +
+THINKING MACHINES LAB
++ You write the learning experiment. Tinker makes the large model + actually move. +
++ It is lower-level than “upload a dataset and receive a model,” + and higher-level than assembling a GPU cluster. Your Python code + owns the data, loss, rewards, rollout environment, and experimental + logic. Thinking Machines owns placement, parallelism, scheduling, + failure recovery, and moving giant tensors through hardware. +
++ That boundary is the product. It keeps the part where research + judgment lives and removes the part where NCCL develops opinions + about your evening. +
+YOUR PROCESS
+TINKER SERVICE
++ Switching from an 8B dense model to a much larger mixture-of-experts + model can be one string change because the hardware layout stays behind + the boundary. +
+Generate candidate tokens from a base model or your current adapter.
+Ask how probable tokens were under a particular policy.
+Evaluate a loss on data and accumulate gradients.
+Use those gradients to update the trainable adapter.
+Freeze a named checkpoint or turn it into a sampling client.
++ Every method is a composition of these verbs. +
++ SFT is next-token prediction over examples you chose. The model sees + a prompt and completion; cross-entropy pushes probability toward the + completion tokens. The intellectual work is mostly upstream: what + counts as a good example, what context is included, and which tokens + are allowed to teach. +
+{sftCode}{datumCode}W′ = W + γBA
++ ORIGINAL WEIGHTS + SMALL LEARNED UPDATE +
++ A model layer contains a large weight matrix W. LoRA keeps + it fixed and learns two skinny matrices, B and A. + Their product is low-rank: it can express a structured change using + far fewer trainable parameters. +
++ Rank controls the dimensionality of that change, not the model’s + intelligence. Too little rank can become a capacity bottleneck. + More rank costs storage and training memory. Thinking Machines’ + experiments found a broad low-regret regime for ordinary + post-training, especially when LoRA is applied across all weight + matrices rather than attention alone. +
++ RL closes a loop. The current policy samples several attempts. An + environment or grader scores them. Advantages express which attempts + were better than their local baseline. Training increases the + probability of better trajectories and decreases the probability of + worse ones. +
+{rlCode}A whole attempt can become an example, a comparison, or a rollout.
+Edits, approvals, reversals, and “that is the wrong frame” can become labels.
+Tests, receipts, task state, and user judgment can supply rewards.
++ The scarce thing is not text. It is trustworthy judgment attached to text. +
++ A run without held-out evaluation is an expensive anecdote. Decide + what improvement means before the first optimizer step, then preserve + examples the model never trains on. +
+Did optimization move in the expected direction?
Did held-out accuracy, reward, or preference rate improve?
Did the model acquire the intended habit rather than a shortcut?
What unrelated ability or style got worse?
TRAJECTORY JUDGMENT / SFT FIRST
++ This is narrow enough to finish and close enough to real agent work + to be diagnostic. It tests the entire Tinker surface without requiring + a synthetic math environment or pretending that reward design is solved. +
+Two attempts at the same task, plus a human choice and one-sentence reason.
Include the task, compact traces, outputs, and receipts. Remove irrelevant noise.
Start with SFT: predict A or B, then explain the decisive evidence.
Especially cases where polish conflicts with correctness or claimed success lacks a receipt.
The useful output is not one score. It is a taxonomy of judgment the model failed to acquire.
+ Expert judgment will transfer where it is visible in concrete labels + and reasons. It will fail where the “expertise” only exists as tacit + context withheld from the training example. +
+Tokenization, masks, batches, loss curves, checkpoints, held-out samples.
+SFT versus preference training on the same underlying judgments.
+Generate fresh trajectories, grade them, and train on-policy.
+Use observed failures to decide what data and reward should exist next.
+