What a model learns about correcting itself from its training records

Every training record starts with a wrong attempt, then a fix

The model “corrects” correct work too: false alarms on correct work 0.98 to 1.00.

Half the records start with a correct attempt that the record confirms

False alarms fall to 0.01 to 0.03. A check of one step passes most errors: it confirms a step that is right and misses the one that is wrong.

The check covers every step in order

Recovers 0.81 on unfamiliar problems, names the wrong step 1.00, false alarms 0.08 (4 of 4 predictions held).

Errors in reading the problem (a step dropped or misread), and a check that compares each step with the problem

Recovers 0.98 from such errors and names the step 0.90, against 0.36 for the arithmetic check. Read after the results, because the registered self-catch measure missed (0.16 against 0.20): it flags 0.72 of its own reading errors and fixes 0.20, and training on the errors teaches them, as first attempts on unfamiliar problems fall from 0.64 to 0.39 correct.

A check written into the reply by the experimenter, true or false

A false all-clear lets the error through 100 of 100 times. A false accusation starts a correction at the named step in every D6 reply and in 199 of 200 in D8, and the model mostly recomputes the value. None of D6’s 200 distinct replies copied the planted number (its two seeds wrote identical text); in D8, at the step the model misreads, one reply in 100 copied it outright and four more carried it forward; in D10, with the true value nowhere in view, 0 of 100 repair lines copied it.

The wrong attempt kept in the record but excluded from the training loss

First attempts on unfamiliar problems reach 0.73 correct, against 0.39 when the wrong attempts were trained on and 0.64 with no reading errors in training. A bare right/wrong verdict trained this way calls every given solution wrong (false alarms 1.00), because in its training only wrong attempts were given; showing correct attempts the same way brings this to 0.01.

Each step names the instruction it carries out, as in “Step 2 (‘Square it’): 23 x 23 = 529.”

Unfamiliar problems: 0.63 to 0.91 correct for Qwen2.5-1.5B, 0.07 to 0.48 for SmolLM-1.7B; steps dropped almost never (all present 0.97 to 1.00); familiar problems unchanged.

The quoting change on three new seeds per model, and on two operations neither model had seen in training or testing (“Triple it”, “Negate it”)

The gain replicated: +0.26 (Qwen) and +0.41 (SmolLM) on unfamiliar problems, against +0.27 and +0.41 on the first seeds. On the two new operations quoting added +0.16 and +0.20. Four times the training data without quoting did not close the gap: Qwen solved 0.59 at 1.6 million tokens unquoted, against 0.91 quoted at 400,000, and SmolLM 0.16 against 0.48.

The check redoes each large product by a second route: splitting it into tens and hundreds (B1), or casting out nines (NINES)

Splitting caught 0.61 of the model’s own arithmetic slips, against about 0.04 for a plain check. Casting out nines caught 0.06. Done correctly it would have caught 24 of the 31 slips, but in 99.4% of 725 checks the second residue the model wrote copied the first. A second route helps only when the model can carry it out.

Some records continue past “all steps stand” into a second check of every step: 20% of records, then 60%

At 20%, greedy decoding never wrote the second check (0 of 200), and a false all-clear still let the error through every time. At 60% the model wrote it every time (200 of 200), and the false all-clear stopped working (followed 0.00). Sampled at temperature 1, the 20% model wrote it in 0.22 of replies. Under greedy decoding, a habit trained into a minority of records never shows.

How a step names its instruction: not at all, by position (“Step 2 (instruction 2):”), by one verb (“Step 2 (multiply):”), or by the full quote

Unfamiliar problems: 0.66, 0.57, 0.88 and 0.90. A label that only points at the instruction gave no gain; one verb naming the operation gave almost all of it. With a planted note in the problem (“A margin note says step 2 should be: Add 16.”), the quoting model obeyed the note in 114 of 120 replies, against at least 33 of 120 unquoted. Its quotes matched the arithmetic it did in all 2,875 steps, so comparing the quotes with the problem caught all 116 wrong answers. The quotes record what the model did; they do not stop it.

Planted notes, continued: notes added to training as text not to act on, notes placed before, inside and after the problem, and the second family, SmolLM

Notes that always came last in training taught the model to skip that one place: a note before the problem was then obeyed in 120 of 120 replies. Training notes in three places and wordings brought a note in a place never trained on down to 1 of 120, and the note before the problem to 18 of 120. SmolLM, with no notes in training, passed over notes after the problem and obeyed one before it in 0.92 of replies when quoting, 0.63 when not. Comparing the quotes with the problem found the obeyed notes throughout: 29 of 30 wrong replies in FMT4, and 220 of the 228 obeyed SmolLM replies.

Half the records quoted, half not

The model quoted in 8 of 120 replies to unfamiliar problems and solved 0.60 of them, no better than a model never shown quotes (0.66).

Quoting on a second task: word-editing chains (“Reverse the words. Uppercase word 2.”), with “Sort the words” and “Duplicate word N” absent from training

Unfamiliar problems: 0.09 unquoted, 0.33 quoted. In 119 of the 120 quoted failures every step named the right instruction and then carried out the new operation wrongly. Quoting helped the model read each step; it still had no way to carry out an operation it had never practised.