Self-correction is competence-gated.

Why it matters

Most proposals for overseeing capable AI rely on verification in some form: cross-checking, debate, self-critique, iterated repair. If verification is competence-gated, these schemes have a structural blind spot. They are weakest on the hard inputs they exist for, because that is where the base model errs and the checker is most likely to be out of its depth as well. There is a second problem: if nominally independent overseers share error basins, adding more of them buys less safety than the count suggests. The same result also reads as a brake on unsupervised self-improvement, since a system that cannot verify beyond its competence cannot safely improve beyond it either.

The three measurements

Cross-model verification is competence-gated. Across 63 task-and-auditor cells spanning seven auditor models, low-competence auditors disagreed with a stronger model'scorrect answers 59–67% of the time. High-competence auditors: 2–3.5%. The correlation between an auditor's own task accuracy and its false-alarm rate is −0.87, and it replicated across three independent runs (−0.86, −0.84, −0.90). A second model is a useful external check inside its own competence region. Outside it, its objections are mostly noise.

Self-repair is foothold-gated. I built a local failure boundary: tasks where best-of-5 sampling stays below 0.7 held-out accuracy. At a matched call budget, execution-feedback self-repair crosses that boundary only where the model has a foothold of fixable near-miss bugs. One task went from 0.0 to 0.69 against a sampling baseline that never solved it, and the crossing replicated. Where there is no foothold, repair does nothing. On one task it regressed, 0.53 to 0.19, because the feedback loop entrenched a wrong approach. I also ran the control this literature often skips: an information-free placebo (same loop, error messages withheld) reproduced most of the boundary crossings. So the claim I will defend is the gating pattern, not that execution feedback is the isolated active ingredient. On my local substrate the mechanism question is underpowered (effect +0.15, 90% CI −0.05 to +0.35), which is one reason the frontier test below matters.

Poison-resistance holds inside the competence region. Given a poisoned retrieved example containing a confident wrong answer, the model reproduced the poison 0 of 72 times. Where it can verify, it computes instead of deferring. Whether poison bites outside the competence region is an open question I have not yet been able to test.

A minimal model predicts all of it

Take a generator that is correct with probability p, a verifier correct with probabilityq, and a shared-error collision probability ρ. This three-parameter model predicts a false-alarm rate of approximately (1 − q). On seven real models spanning the competence range, the observed fit is −0.01 + 0.83·(1 − q), against a predicted intercept of 0 and slope of 1. The same model predicts a catch rate of 1 − (1 − q)·ρ. Measured ρ is high: when generator and auditor are both wrong, they produce the same wrong answer about 55% of the time, and the catch-rate prediction holds at correlation 0.83. On this evidence, "independent overseers" is the wrong frame. The effective number of independent verifiers is well below the nominal count.

From law to deployment rule

None of this argues against using verifiers. It prices them. In the model, a verifier-plus-escalation layer improves accuracy in nearly every (p, q, ρ) configuration, but the gain per unit cost peaks when a weak generator is paired with a strong verifier that has low error overlap, and it collapses for strong generators or high-overlap pairs. The practical rule: invoke a model verifier when its skill clears a computable break-even threshold and its error overlap with the generator is low; otherwise escalate to an external check. The scope condition is measurable, which is the point.

Relation to prior work

The negative headline is consistent with Huang et al. (Large Language Models Cannot Self-Correct Reasoning Yet) and Kamoi et al. (When Can LLMs Actually Correct Their Own Mistakes?). What this study adds is the scope law: when checking helps and when it does not, derived from a minimal model and replicated, rather than catalogued case by case. The matched-budget-plus-placebo design is a measurement caution for the self-refine line of work (Madaan et al.), where iteration gains and feedback gains are easy to conflate. Relative to weak-to-strong generalization (Burns et al.), it quantifies when a weak verifier's signal is net-negative. My prover-verifier run is an honest local negative for adversarial-checking schemes (Irving et al.; Kirchner et al.): on this substrate it reduced accuracy. The shared-basin result extends the LLM-judge self-preference concern (Panickssery et al.) from self-preference to family-correlated judgment.

Method

The experiments run continuously on a harness I built for this program. Every claim carries a pre-registered minimum effect of interest, an anti-claim written before the run, a 90% bootstrap confidence interval, and a four-way verdict. Discovery runs are kept separate from confirmation runs, and refuted claims go to a graveyard file rather than quietly disappearing. That standard has refuted four of my own headline hypotheses mid-program, in writing. I trust the surviving results because the same machinery killed the others.

Limitations

One open-weight 32B coder model. Constructable number-theory, combinatorics, and small-graph tasks with programmatic oracles. A single consumer GPU. The local failure boundary is small (~8 tasks), so mechanism-isolation tests are power-limited by construction. The strong version of "structure beats scale at the failure boundary" is a well-specified frontier hypothesis, not a local conclusion. The absolute break-even threshold depends on base rates; what I expect to transfer is the sign, slope, and strength of the gating, not any single number.

Status and next step (July 2026)

The local pilot is complete and replicated. The two instruments, the auditor-competence protocol and the boundary-repair harness, are built and ready to port to frontier models, where the failure region is large enough to power the tests my local substrate cannot. The near-term plan is a preregistered frontier validation: run the competence-region protocol on a public frontier model, measure whether the false-alarm law and the shared-basin collision rate replicate, and publish the outcome either way. The cross-vendor question interests me most: do frontier models from different labs share error basins the way my local model families do? Ensemble-oversight schemes assume they do not, and I have not found a published measurement. A negative result here would still be worth having. It would say how much trust verification-based oversight can carry at capability.