Boundary Intelligence
A multi-project program (986+ pre-registered trials) finding that simple mechanisms at system boundaries can match or beat far larger internal-model approaches. Reported with its honest negatives.
Synthesis · 986+ trialsIndependent AI-safety research
Legible AI: when can a model check its own work?
I built a small lab to measure when an AI can catch its own mistakes. So far the answer is that self-correction is competence-gated: a model can only check what it already half-knows, and the checking fails on exactly the hard cases where oversight needs it. I derived the pattern from a minimal model, confirmed it on seven real ones, and worked out what it implies for when a verifier is worth deploying.
The finding
Self-correction is competence-gated. Verification fails where it's needed most.
Most plans for overseeing AI lean on one move: use another AI to check the first one. I measured when that move works. A checker helps inside its own competence region, and outside it, its objections are mostly noise. The models also turn out to be less independent than they look. When two of them are wrong, they hand you the same wrong answer more than half the time, like students who studied from the same textbook. I derived this from a minimal model first, then confirmed it on seven real ones, then replicated it on fresh tasks.
−0.87correlation between an auditor's competence and its false-alarm rate on correct work — low-competence auditors flag 59–67% of right answers vs 2–3.5% for competent ones (63 cells, seven models; replicated −0.86 / −0.84 / −0.90)
ρ ≈ 0.55how often "independent" models give the same wrong answer when both are wrong — a verifier's catch rate tracks the derived law 1 − (1 − q)·ρ at corr 0.83
0.0 → 0.69held-out accuracy on a task best-of-N sampling never solved — self-repair at a matched call budget, replicated; it works only where the model has a foothold, and can regress where it doesn’t
Scope, stated up front: one open-weight 32B model, tasks a program can grade, a single GPU in my office. I claim the measurements, not more than that. The rigor standard behind them has refuted four of my own headline hypotheses in writing, so the surviving results have earned some trust. Methods, result cards, and the full writeup are on the research page, and I'm glad to share the underlying evidence directly.
Selected work
Research and the tools it produced
Lattice
A local, OpenAI-compatible execution gateway (~13,300 LOC, 133 tests): tiered model routing, async task management, and a containment validator that caught 100% of adversarial proposals in testing.
Working softwarelattice-commit
A git-native, test-gated wrapper for LLM code-repair loops. Restores reliable multi-file repair where a bare loop silently reverts good fixes. Packaged and benchmarked on 22 workspaces.
Alpha · PyPIGovernor
A keyword classifier at the request boundary that routes each turn to the cheapest adequate model tier — 23–65% cost reduction at 95–96% quality in measured sessions.
Open source · MITThe tool repositories — governor andlattice-commit — are public. Research code, data, and reproduction bundles are available to reviewers and collaborators on request.
About
Independent research, run under a written rigor standard
I'm an independent researcher running a focused program on a single workstation GPU. The work is deliberately small and falsifiable. Expectations are pre-registered, every claim has kill criteria, and the uncomfortable controls get run even when they threaten the thesis. When I get something wrong, the record says so: four of my own headline hypotheses have been refuted by my own harness, and the retractions are written down.
The through-line is legibility: mechanisms a human can read, audit, and reason about. I think oversight you can't read is oversight you can't trust, and I'd rather demonstrate that on narrow, checkable problems than assert it broadly.
How a sustainability educator ended up working on legible AI →
Work with me
Open to research collaboration, funding conversations, and reviewer access to the full evidence.
bentleymoonperkins@gmail.com