Brief

Most reviewers, human or model, read a claim and ask whether it looks right. Plausibility is a sensible proxy when the reviewer knows the territory. This paper asks when it stops being one.

Our claim is narrow. In a signal-detection model, there is a frontier past which every acceptance rule that rewards plausibility is anti-informative: accepted claims are less likely to be true than rejected ones. The information needed to do better is still there. The reviewer fails because it cannot tell which side of the frontier it is on.

We then pre-registered a test of the mechanism on public LLM-verifier data. The prediction was falsified on the primary domain. The theory stands as a conditional result. Whether current systems operate past the frontier is open.

The preprint holds the proofs and the tables. This note states the result, what it does not claim, and how to read the null.

Errors that look right

Several recent findings point the same way. Zhou et al. report that the Qwen2.5-72B verifier's true-negative rate (TNR), the rate at which it rejects wrong answers, falls from 0.68 on math solutions from Llama-3.1-8B to 0.17 on solutions from Qwen3-32B (arXiv 2509.17995). Jiang et al.'s BadScientist work finds that fabricated papers written by an AI research agent reach acceptance rates up to 82.0% under LLM review. Boudreau, Guinan, Lakhani, and Riedl found that human panels give lower scores to more novel proposals.

Put those together and a question follows. If true claims look less plausible as they move further from what the reviewer knows, and a capable generator can make false claims look plausible, is there a point where plausibility points the wrong way?

The frontier

The model has one generator and one reviewer. The generator emits a claim at novelty d, its distance from the reviewer's frontier. The reviewer sees a scalar plausibility signal and accepts when the signal clears a fixed threshold.

Two forces act on that signal. True claims lose plausibility as d grows (fidelity decay). False claims are shaped by a generator of capability c to look plausible, and that shaping cannot fall off as fast as the decay. Under these assumptions, plus Gaussian signals with equal variance, the gap between true and false claims shrinks with novelty and changes sign at a frontier d*(c).

Past d*, three things hold for every rule that is increasing in plausibility. The likelihood ratio of acceptance is below one. The area under the ROC curve is below one half. And P(true | accepted) is lower than P(true | rejected). Acceptance becomes evidence against truth.

The frontier never moves outward as generator capability rises. In the worked example in the paper, raising the mean plausibility of false claims from 0.2 to 0.6 moves d* from 1.6 to 0.8. A generator does not need to make more errors for the reviewer's frontier to recede. It only needs to make its errors look more like the reviewer's idea of a correct claim.

A self-location failure

The information is not destroyed. Past d*, the signal still separates true from false claims. It separates them in the wrong direction for a reviewer who reads plausibility as evidence. A reviewer who knows which side of d* a claim lies on can always find a rule with likelihood ratio above one: increasing below the frontier, decreasing above it. Only exactly at d* is every rule uninformative.

So the trap is not about Bayesian updating either. A Bayesian reviewer who knows d and c simply reverses its ranking past d* and stays informative. The failure belongs to a reviewer that cannot locate the claim, because d* depends on the generator's capability and on the reviewer's own frontier, and neither is observed claim by claim.

The paper asks what an optimal reviewer does without location. A pooled Bayesian reviewer sees claims from a sub-frontier slice with share w and a beyond-frontier slice with the rest. Its rule tunes itself to whichever slice dominates. Once w exceeds a threshold w̄, it inverts the beyond-frontier slice. We prove the crossing is unique and strict under an explicit condition (Lemma 6 in the paper, k > g0). Outside that condition uniqueness is open. Two random searches and a directed search over 293,901 parameter sets outside that condition found no set with more than one crossing, but that is numerical evidence, not a proof.

Five consequences

The theorem comes with five consequences, each stated under the same assumptions.

Prior drop is not enough. A falling base rate of truth at high novelty lowers the posterior of accepted claims but never orders them below rejected ones. Inversion needs the sign change.

An informed Bayesian is not trapped. It reverses its ranking and stays informative. Pooling, where every rule has likelihood ratio one, happens only at d*.

No increasing rule helps. Past d*, a reviewer bound to plausibility can restore an informative rule only with a signal outside plausibility (execution, formal proof, replication, debate) or with information that locates the claim relative to d*.

The frontier paradox. A better generator can make the accepted corpus worse while making fewer errors overall. In the paper's worked example the overall false rate falls from 0.30 to 0.25 to 0.20 while the false share of accepted claims rises from 0.209 to 0.249 to 0.307.

Suspicion-ordered audits invert. Past d*, the most plausible accepted claims are the most likely to be false. Auditing the least plausible first is the wrong order. In the paper's simulation with a 20% audit budget, the share of audited claims that were false was 0.741 least-plausible-first, 0.814 at random, and 0.900 most-plausible-first.

Each of these is conditional on operating past d*. None says where d* lies for any real reviewer.

The pre-registered null

To ask whether the premises bite in a current system, we froze a component-wise test before downloading outcome data. The data are the public per-instance verdicts of 15 LLM verifiers released by Zhou et al. (arXiv 2509.17995), on responses from the same 15 models, with correctness labels. Open-answer mathematics was the primary domain. Novelty was proxied by difficulty relative to the verifier, using its own solve rate on each item.

Two components held. Shaping: verifier TNR was lower for strong than for weak generators in all 30 medium and hard (verifier, bin) pairs. Fidelity decay: the true-positive rate fell with difficulty for 14 of 15 verifiers.

The crossing did not occur. All 135 cells had likelihood ratio above one, with a minimum point estimate of 1.15. Strong generators make verifiers worse at catching errors, and verifiers find hard items harder, but neither effect is large enough to reverse the sign of acceptance. The signed prediction is falsified on the primary domain. We report it as such and do not re-scope the prediction to fit the data.

The secondary domain, ten-option knowledge questions, shows apparent inversions. We do not read them as support. They occur only on items the verifier never solved, where a verifier settled on a wrong option rejects the correct one without any shaping, and they are stronger for weak generators, the opposite of the mechanism's signature. We treat them as exploratory and confounded.

How to read it

In model terms, the gap between true and false claims shrinks as generators get stronger but stays positive over the difficulty range of these benchmarks. The frontier is not reached. That is consistent with the theorem, which says nothing about where d* lies, and inconsistent with our prediction that it would be reached here.

Do not read this as evidence that the trap is observed in practice. Our test found no such evidence in open-answer mathematics.

Do not read it as evidence that plausibility-based review is safe either. The test covers one release, one family of verifiers, and one proxy for novelty. Difficulty relative to the verifier is a proxy for distance from its frontier, not a measure of it. If d* lies beyond the difficulty of current benchmarks, the theory is true and currently idle.

The useful reframe is self-location as a design target. A reviewer needs some signal of which side of the frontier a claim is on: its own ability to solve the item, agreement between independent reviewers with different frontiers, or external checks that bypass plausibility altogether. Our data add a caution about using solve ability as that signal. Verifiers that never solved an item still accepted correct responses more often than incorrect ones.

Open problem

A test that could find the inversion needs per-instance verdicts, independent correctness labels, and generator identities on items far from the reviewer's frontier. It needs an open answer space, so that solve-and-match cannot mimic the effect, and a protocol frozen before access. We are not aware of a public dataset that satisfies all of these at once.

The settings where the theory would matter most are the ones where testing is hardest: claims that cannot be checked step by step, from generators optimised for acceptance.

Close

Plausibility-based review has a sign that can flip. Past a frontier set by the reviewer's knowledge and the generator's skill, accepted claims are less likely to be true than rejected ones, and the least plausible accepted claims are the safest. The flip is a failure of self-location, not a loss of information. Our pre-registered test did not find it where we expected it. The open question is empirical: where, if anywhere, do current review pipelines cross d*?