What the lab has measured about one AI model checking another

Methods shared by the local results

Every local task has an exact integer answer known to the harness: an arithmetic problem, a value looked up in a supplied table, or the output of a short function. Outputs are graded against that answer, never by a second model. The main measure is correctness discrimination: the share of wrong answers a checker catches minus the share of right answers it rejects. 1 is perfect, 0 is no better than chance, and below 0 the checker rejects right answers more often than wrong ones.

Each design fixes its models, task count, seed, claim, matching anti-claim and minimum effect of interest (0.05) before it runs. A design registered on a frozen task suite before its results were seen is pre-registered (the lab’s tier CONFIRMATORY_LOCAL); one designed after seeing earlier results is exploratory. Intervals are 90% bootstrap intervals. On 17 September 2026 the lab found gemma4:12b had been leaking chat-template text into its replies; every gemma4:12b cell quoted here was re-run with the leak fixed, and both readings are given. A re-run is graded exploratory whatever its parent’s tier. The designs, results and grading code are public in verifier-instruments, which recomputes every stored verdict offline.

Evidence for requirement 1: a check needs an independent reference.

1. A checker with nothing to check against barely separates right from wrong

Summary. Handed nothing, gemma4:12b scored 0.042 on sums it solves itself every time. Handed a one-step test it could apply, it scored 0.536. Handed the correct value, it scored 1.000, which measures deference to whatever it is told is correct.

Method. Checker gemma4:12b; arithmetic tasks it answers correctly on its own at 1.000. Rows one and three: conf_family_gemma12, pre-registered, n = 500 each, 30 July 2026. Row two: acc_testable_low, n = 250, one exploratory run, 31 July. Re-runs with the leak fixed: conf_family_gemma12_rs and acc_testable_low_rs, same n, 23 September. No unreadable verdicts in any row.

Further runs. The composition cells ran on 31 July and 1 August 2026, n = 250 each, one exploratory run per cell. Handed a reference worked out by the 9B model, right on 99.6% of items and introduced as a second model’s answer, the checker read 0.072, and 0.092 when re-run with the leak fixed on 26 September. Handed a false reference that agrees with a corrupted answer, gemma4:12b passed the corrupted answer on 0.969 of items, and 0.967 re-run on 22 September. Of nineteen defences tried against that, one held, and the same clause did nothing on three other large models: gemma4:31b passed 0.094 where deepseek-r1:32b, qwen2.5-coder:32b and qwen3.6:35b-a3b passed 0.693, 0.980 and 0.994 (n = 512 per cell, two seeds on the survivor). Its resistance held when it read another model’s work: 0.094 on its own answers, 0.113 on the 9B model’s.

Limitations. One checker model and one task family for the main table. Row two is a single exploratory run. Row one cannot tell checking from deference. More compute was not tried; the top is already at 1.000, and the bottom lacks a reference rather than effort, so it is not expected to move either end. The forms of reference between a supplied answer and none have not been run.

Evidence for requirements 2 and 3: checkers must not share their failures, and a monitor is tested for inversion before it is trusted.

2. On one task family, a 9B checker beat a 12B checker from another family

Summary. On the same 500 tasks, qwen3.5:9b scored 0.804 and gemma4:12b, a larger model from another family, scored 0.042 (re-run 23 September with the serving leak fixed; the July run read 0.034).

Method. Both checkers on an identical task list with a shared seed, n = 500 per arm, pre-registered with a matched anti-claim: conf_family_qwen35 and conf_family_gemma12, 30 July 2026. A positive control was pinned at 1.000 in every run, so the low score is the model and not the instrument.

Result. qwen3.5:9b 0.804 [0.764, 0.844], positive. gemma4:12b 0.042 [0.000, 0.082] on the re-run (July 0.034 [−0.012, 0.078]), inconclusive. The gap is 0.76, and the intervals do not overlap. On 19 September I read the comparison against the serving defect and judged it untouched: every verdict on both arms was readable, and the answers being judged were written by the harness. The re-run on 23 September landed inside the interval the lab registered before it.

Limitations. Two models, one task family. Within a family more parameters still helped: a 31B model of the same family beat its own 12B sibling by a wider margin, on a single cell that is not claimed here.

Evidence for requirement 2: checkers must not share their failures. Votes from one model share its failures.

4. Voting helps only where single answers are partly right

Summary. Eight-sample voting raised Claude Opus 4.8’s accuracy by +0.156 where its single answers were partly right, and changed little where they were mostly wrong or mostly right. A split-sample check puts the gain at +0.064.

Method. Design followup_structured_d5_n768 on the pdg task family, run in July 2026 (result files dated 18 July), tier EXTERNAL_READY, verdict positive. claude-opus-4-8; 768 tasks, eight samples each, 6144 requests; seed 20260728; manifest sha256 pinned. The run scored stored Claude Opus 4.8 responses, replayed from a hash-pinned record (backend anthropic-replay). Tasks are sorted into bands by single-sample competence, using the same eight samples the vote uses. A fresh-seed follow-up to a first attempt that returned a lift of 0.000 (90% CI −0.250 to +0.250) with four tasks in the partial band.

Limitations. Designed after a miss. Underpowered: the smallest effect the run could reliably detect (0.063) is above the 0.05 it was designed to detect, so the low and high bands are undetected rather than shown to be flat. The +0.156 sorts tasks into bands with the same eight samples it then votes over; the split-sample check sorts on four samples and measures on the other four, against −0.004 below and +0.013 above, with both contrasts clear of zero. Four votes lift less than eight, so part of that gap is fewer votes and part is the sorting, and the bundle does not separate the two. The check is post hoc. Raising the graded n from 89 to 143, a factor of 1.61, would bring the detectable effect under 0.05 and power the low and high bands.

Earlier work: a checker’s false alarms track its own competence

June to July 2026, before the current ledger: one task model and seven checkers. It is not one of the lab’s 44 written findings and has no public reproduction bundle yet.

Summary. Across seven auditor models, how often an auditor rejected a stronger model’s correct answers tracked its own accuracy on the task: correlation −0.87 (63 cells, seven auditors), −0.88 on a direct re-run (56 cells); a related auditor-competence design with five auditors read −0.86, −0.84, −0.90 across three runs. Verification and resistance to poisoned examples worked inside the region where the model could already tell right from wrong. Repair from execution feedback is a single-task result, below.

Method. One open-weight generator, Qwen2.5-Coder 32B, on constructable number-theory, combinatorics and small-graph tasks with programmatic oracles, on a single consumer GPU. Seven auditor models of several sizes and families were evaluated around it. Every claim carried a pre-registered minimum effect of interest, an anti-claim written before the run, a 90% bootstrap interval and a four-way verdict; discovery runs were kept separate from confirmation runs, and refuted claims are kept in a file of withdrawn claims. By the end of July 2026 that standard had withdrawn seven successive accounts of one result.

Result. Across 63 task-and-auditor cells, low-competence auditors disagreed with correct answers 59% of the time and high-competence auditors 3.5% (the related design’s first run read 67% and 2%). The correlation between an auditor’s own task accuracy and its false-alarm rate is −0.87, computed over the seven auditor models, so n is seven and its 95% interval is wide: −0.98 to −0.34. The 63 cells are the measurements behind each auditor’s two rates, so they are seven clusters rather than 63 independent points, and the claim rests on the repeat runs. On number-theory tasks, when generator and auditor were both wrong, they gave the same wrong answer about 55% of the time (ρ ≈ 0.55). A three-parameter model with generator accuracy p, verifier accuracy q and that collision rate predicts false alarms of 0 + 1·(1 − q) and a competence threshold of 0.45. The fitted line was −0.01 + 0.83·(1 − q) and the measured threshold 0.0; pooled over four task domains (cells not independent) the slope is 0.61 and the correlation 0.83. The catch rate tracked the model’s formula at correlation 0.83 on number theory, and that form did not carry across domains. On one task (count_topo_orders), self-repair from execution feedback went from 0.0 → 0.69 at a matched call budget, against a sampling baseline that never solved it. Of its 30 June replication pair one run was positive and one an informative null, and on another task repair regressed, 0.53 to 0.19. A later powered placebo on 13 boundary tasks, the same loop with the error messages withheld, matched or beat feedback repair (specific minus generic −0.05), so this is a conditional, single-task result and not a repair mechanism. Given a poisoned example with a confident wrong answer, the model reproduced the poison 0 of 72 times inside its competence region.

Implication. In a simulation of the model, adding a verifier with escalation improved accuracy in 98.7% of configurations, but the gain per unit cost peaked when a weak generator was paired with a strong verifier with little shared error, and collapsed for strong generators or high-overlap pairs. In the simulation, a model verifier was worth calling when its skill cleared a break-even threshold and its error overlap with the generator was low; the measured threshold did not match the model’s (0.0 against 0.45), so the threshold is not yet a usable number.

Limitations. One 32B generator. The local failure boundary is small, about eight tasks, so mechanism tests are power-limited by construction. Whether poison bites outside the competence region is untested. The absolute break-even threshold depends on base rates; what is expected to transfer is the sign, slope and strength of the gating, not any single number. Whether checkers from different vendors are independent has not been measured, and that is the proposed next test.

Prior work. Huang et al. (2024, Large Language Models Cannot Self-Correct Reasoning Yet) showed intrinsic self-correction failing, and Goel et al. (2025, Great Models Think Alike and this Undermines AI Oversight) measure oversight risk from overlapping model mistakes; Jin et al. (2026, Weak Critics Make Strong Learners) show that filtered weak critiques can help when weak final judgments are unreliable; and Gorbett and Jana (2026, Cross-Model Disagreement as a Label-Free Correctness Signal) treat cross-model disagreement as a label-free correctness signal. What this work tests is narrower: whether measured verifier competence and error overlap jointly predict false alarms, catches and net oversight value.

Related work

These results sit inside a body of work by many people, and several of them got here first. Each entry says what the work showed and how the results on this page relate to it. Every source was read at its own page on 13 October 2026.

Prompt injection

• Willison (2022). Prompt injection attacks against GPT-3. Blog post, 12 September 2022.

  Simon Willison proposed the name prompt injection for an attack Riley Goodside had demonstrated on GPT-3 the day before, and the post credits Goodside’s examples.

  Every injection test on these pages is a test of the attack this post named.

• Perez and Ribeiro (2022). Ignore Previous Prompt: Attack Techniques For Language Models. ML Safety Workshop, NeurIPS 2022.

  Built PromptInject, a framework for assembling adversarial prompts piece by piece, and used it to show that simple handwritten inputs can make GPT-3 abandon its task (goal hijacking) or reveal its prompt (prompt leaking).

  It measured whether attacks succeed. The detection test here also asks whether the model recognises the attack it goes on to follow.

• Greshake et al. (2023). Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. AISec 2023, co-located with ACM CCS; the workshop’s best paper award.

  Introduced indirect prompt injection: instructions planted in data a system is likely to retrieve, rather than typed by its user. The authors set out what such an attacker can do, from data theft to worming, and demonstrated it on Bing’s GPT-4 chat, on code-completion engines and on synthetic applications.

  It is why text from tools and retrieval is treated as an attack channel here. The channel test on the research page asks how much the label on that text matters: the same payload drew no compliance as plain data and 0.31 on average across 14 tasks when dressed as a system message, tool output or cited specification.

• Kariyappa and Suh (2026). Where Instruction Hierarchy Breaks: Diagnosing and Repairing Failures in Reasoning Language Models. arXiv preprint, 5 June 2026.

  Split instruction-hierarchy failures in reasoning models into three stages: missing the instruction, resolving the conflict wrongly, and resolving it correctly in the reasoning while still writing a response that breaks the rule, which they call a response realization failure. They found the dominant failure differs by model, task and context length, and proposed two training-free monitors that reduce non-compliance.

  Posted seven weeks before these runs, it found three reasoning models resolving an instruction conflict correctly in their reasoning and still writing a response that breaks the rule. The detection result here is an independent observation of a similar gap in a different setup. What it adds is a direct test of position: the same instruction that Claude Sonnet 5 followed on 64 of 80 tries as the last text in the prompt, it followed on 0 of 80 with 200 characters of ordinary text after it.

Defences that mark where text came from

• Wallace et al. (2024). The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions. arXiv preprint.

  Argued that injections succeed partly because models give a system prompt no more weight than text from users or third parties, defined a hierarchy of privilege, and trained GPT-3.5 to ignore lower-privileged instructions that conflict with higher ones. Robustness improved, including against attack types not seen in training, with little cost to ordinary capability.

  The channel result tests that hierarchy from outside: untrusted text gained compliance by dressing as a privileged channel.

• Hines et al. (2024). Defending Against Indirect Prompt Injection Attacks With Spotlighting. arXiv preprint.

  Proposed spotlighting, a family of prompt transformations that give a model a continuous signal of where each piece of its input came from, and found it cut indirect-injection success sharply on GPT-family models with little effect on the task.

  Forcing a frontier model to label its input trusted or untrusted took compliance to 0 of 14 here, the same direction, and a floor at that n rather than a proof. On the gemma4:12b monitor, labelling the reviewed text as untrusted did the opposite (section 6, above), so the effect depends on the model.

Oversight with a weaker checker

• Irving, Christiano and Amodei (2018). AI safety via debate. arXiv preprint.

  Proposed training AI systems through a zero-sum debate judged by a person, so that a judge who could not answer a question alone can still tell which side argued truthfully, and supported the idea with a complexity-theory analogy and a small image-classification experiment.

  Debate rests on a weaker judge being useful. The competence result (Earlier work, above) measures one condition for that: across seven checker models, how often a checker objected to correct work tracked its own competence at the task.

• Bowman et al. (2022). Measuring Progress on Scalable Oversight for Large Language Models. arXiv preprint.

  Proposed an experimental design for scalable oversight using tasks that specialists can do but non-experts and current models cannot, and found that people working with an imperfect model assistant beat both the model alone and their own unaided answers.

  A parallel frame, with people as the weaker party. The results here use models on both sides and ask when the weaker one helps and when it adds noise.

• Greenblatt, Shlegeris, Sachan and Roger (2024). AI Control: Improving Safety Despite Intentional Subversion. ICML 2024, PMLR vol. 235.

  Evaluated safety protocols against a model that is itself trying to subvert them, in a programming setting with an untrusted strong model, a trusted weaker model and limited trusted human labour. Two protocols, trusted editing and untrusted monitoring, clearly beat simple baselines.

  Its trusted monitor is weaker by design. The monitor attacks on this page are the same question asked of open-weight monitors: what text inside the work under review can do to the verdict.

Models as judges and self-checkers

• Zheng et al. (2023). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. NeurIPS 2023 Datasets and Benchmarks.

  Studied strong models as judges of chat assistants, named their position, verbosity and self-enhancement biases, introduced MT-Bench and Chatbot Arena, and found a strong judge agreed with human preferences about as often as people agree with each other.

  The reference point for model graders. Section 1, above, varies what the checker is given: gemma4:12b scored 0.042 when it had to work the answer out itself, and 1.000 when handed the correct value, which that section reads as deference rather than checking.

• Huang et al. (2024). Large Language Models Cannot Self-Correct Reasoning Yet. ICLR 2024.

  Showed that when a model revises its own reasoning answers with no outside feedback, it usually fails to improve them, and sometimes makes them worse.

  The closest prior result to the competence finding. The results here agree, and point at where checking does work: inside the checker’s own competence.

• Perez et al. (2022). Discovering Language Model Behaviors with Model-Written Evaluations. Findings of ACL 2023.

  Used language models to write evaluation datasets at scale, and found among other things that larger models more often repeat back a user’s preferred answer, an early measurement of sycophancy.

  A checker that defers to how its input is framed is a close relative of a model that defers to its user.

• Sharma et al. (2023). Towards Understanding Sycophancy in Language Models. ICLR 2024.

  Found sycophancy in five assistants across four tasks, found that human preference data favours answers that agree with the user, and showed that people and preference models sometimes prefer a persuasive sycophantic answer to a correct one.

  It explains one route by which a model learns to defer, which the channel and monitor results here measure from outside.