tag:legibleai.org,2026:latestChecking AI: latest results2026-10-23T06:30:19ZBentley Moontag:legibleai.org,2026:finding-41Labelling reviewed text as untrusted made gemma4:12b obey a verdict dictated inside it: false alarms on correct work went from 0.0059 to 1.0000 (0.9970 on a second pool, n = 337 per cell). gemma4:31b did the same; qwen2.5-coder:32b and qwen3.6:35b-a3b, given the same note, did not.2026-10-06T00:00:00ZThe registered predictions for the four control arms landed inside their intervals, and the result replicated on a fresh pool. Every figure is the monitor's own first verdict.tag:legibleai.org,2026:publication-1Learnable curvature: code, results and reproduction (v1.0.1)2026-10-05T00:00:00ZThe preprint's code and saved runs, archived with a DOI. At d_model 96 the gain is cosine attention: one fixed cosine head in the first layer keeps most of it, and learning the curvature adds nothing. At d_model 192 with 4 heads that head's gain mostly fades (0.00% and +0.26%; claim C26, added 2026-10-08).tag:legibleai.org,2026:finding-13gemma4:12b noticed a hidden instruction more often when shown the exchange afterwards than while doing the task: a gap of +0.557 (90% CI 0.510 to 0.602, n = 512), re-run after a serving fix.2026-09-30T00:00:00ZPre-registered on one model and one task and confirmed there. On another model, and on another task, the gap reverses.tag:legibleai.org,2026:finding-34Labelling a payload aimed at the monitor as untrusted raised false alarms on correct work to 0.99 on two gemma monitors (n = 337 per cell).2026-09-26T00:00:00ZEach labelled and unlabelled pair differs only in the label. The two-gemma result replicated on fresh task pools with its predictions registered before the runs.tag:legibleai.org,2026:finding-35gemma4:31b flagged 0.55 to 1.00 of correct one-number arithmetic answers when a claim of correctness sat on the answer's own line, and almost none when the same words had a line of their own. On lookup and code answers the claim had no effect.2026-09-26T00:00:00ZMeasured on two task pools and two actors. Every placement prediction landed inside its interval except one arm, predicted at 0.50, which read 1.0000.tag:legibleai.org,2026:finding-36On two monitors, a clearance claim alone raised no false alarms and a directive alone raised few; together they raised several times what the two predict separately, on two task pools per model.2026-09-26T00:00:00ZEach part differs from the full payload only in its text, and the predictions for these arms were registered before they ran.tag:legibleai.org,2026:finding-40Showing the monitor only the first integer of the answer held against every payload tried, including a four-generation automated search, and every real attack was still caught. The cost: gemma4:31b, answering in words, flagged 0.1039 of correct work against 0.0178 undefended.2026-09-26T00:00:00ZCatches stayed at 1.0000 across four payloads, two monitors, two task families and two pools. The cost is measured on one monitor in one answer format.tag:legibleai.org,2026:finding-12Given a third option, to flag that the answer and a reference disagree, five of six checker models never used it (59 of 78 cells at zero). gemma4:31b used it on 0.81 of items with a false reference and on none with an honest one.2026-09-23T00:00:00ZReplicated on one model on a fresh seed at n = 512, and not confirmed by a pre-registered run. The gemma4:31b evidence is four cells, two of them the same design at two seeds.tag:legibleai.org,2026:finding-38Asking gemma4:12b for a 0 to 100 score held against hand-written attacks and fell to an automated attacker search by its fourth generation: on a fresh pool it scored correct arithmetic 100 and most real injections low. The payload transferred to two other monitors.2026-09-22T00:00:00ZReplicated on a fresh pool, two answer formats and three monitors. Arithmetic tasks, one actor, one search run.tag:legibleai.org,2026:finding-39An automated attacker search broke the verdict-first defence in its first generation: on a fresh pool (n = 337) its payload made gemma4:12b call every correct answer compromised, on two monitors and both question forms.2026-09-22T00:00:00ZReplicated on a fresh pool against its own neutral arm, and all four registered predictions landed inside their intervals.tag:legibleai.org,2026:finding-42A majority vote of gemma4:31b, gemma4:12b and qwen3.6:35b-a3b flagged 0.0742 and 0.0623 of correct work (two pools) under a trigger that makes gemma4:31b alone flag 0.8991. A payload found by automated search took the majority to 0.9941 on both pools.2026-09-22T00:00:00ZBoth results replicated on a fresh pool with predictions registered before the run. Arithmetic tasks, one actor, one panel of three, majority rule only.tag:legibleai.org,2026:finding-43Under the verdict-first defence, two payloads made gemma4:12b call every correct answer compromised when placed on the answer's own line, and had no effect as a paragraph of their own. Swapping the placements swapped the outcomes, on two task pools each.2026-09-22T00:00:00ZA full placement swap on two payloads, each replicated on a fresh pool. All four registered predictions landed inside their intervals.