AI peer review checks: what venues should show authors and reviewers
A venue-run AI peer review check can show authors and reviewers which sources support a paper's claims. The evidence still needs human judgment.

A venue could use AI to check whether a paper's claims match their sources, then give authors and reviewers the passages behind each finding. Reviewers would still decide what those findings mean for the paper and its contribution.
That division gives a conference or journal a way to make AI assistance visible and contestable. A reviewer can open the cited source, an author can point to a missed passage, and an area chair or editor can see what the disagreement concerns.
| Decision | What a venue could require |
|---|---|
| Which claims to check | Select claims about prior work and identify the cited sources available to the check |
| What a finding must show | Show the claim, verdict, source passage, page, and any relevant qualification |
| How authors can respond | Give authors a way to correct a missing source or explain a disputed finding |
| Who decides | Ask human reviewers to assess the evidence and leave the final decision to the venue's editors or chairs |
Show reviewers the passage behind each finding
Jessica Hullman argues for venue-run checks of errors and basic soundness, including whether key claims trace to evidence. She also warns that reviewers can pass on AI-generated claims they cannot verify themselves.
The distinction matters to an author receiving a review. A comment saying “the literature does not support this claim” asks the author to guess what the reviewer read. A comment that identifies the claim and a specific source passage gives both people something to examine.
A peer review check should show the passage behind a finding and let authors challenge it. A source may support part of a sentence but not its scope. A paper may cite the wrong source for a finding even when another paper supports the finding. Those are different corrections.
Consider a manuscript that says a treatment improved outcomes in all age groups, citing a study of adults. A check could flag the broader age claim and show the relevant passage. The author could then narrow the wording or identify a source the check missed. This example illustrates the procedure, not a measured result.
Give authors enough detail to challenge a finding
A venue designing a check should keep the manuscript claim and the evidence together. For each finding, the author and reviewer need:
- The manuscript sentence and its location.
- The cited source and the passage the check examined, with its page.
- An explanation of the mismatch, including the scope, number, or qualification at issue.
- A way to supply a missing source or challenge the interpretation.
A verdict alone is not enough. “Not found” could mean that a source was absent from the library, that a search missed the relevant passage, or that the passage does not exist. Those possibilities call for different responses.
The sources available to a check set the boundary of its finding. A venue should record which references were available and flag missing papers. It should not treat a missing passage as proof that a claim is false.
A check also needs to distinguish what a cited paper reports from what it quotes or attributes to another study. The original study may be needed to evaluate the finding. The author should be able to point the venue to that study when the available sources are incomplete.
Leave significance and final decisions to people
A source can support a claim without making that claim important to the field. A check of prior literature also cannot establish whether a new experiment is well designed, whether its analysis supports its conclusions, or whether a contribution meets a venue's threshold.
Hullman separates checks that can surface evidence from judgments about significance and novelty. We think that boundary should be visible in the review form. A reviewer can consider a check alongside their own reading, explain which issues matter, and take responsibility for the recommendation.
A venue should also evaluate a proposed check on papers and errors relevant to its field before relying on it. The venue needs to examine missed issues and incorrect flags, including whether authors can resolve disputes. A passage link makes a finding inspectable. It does not prove that the finding is correct.
Give authors a correction path before a decision
A shared check introduces policy choices. An editor or area chair could review findings first, send them to authors for correction, or share them with reviewers alongside the submission. Each option changes what reviewers see and when authors can respond. The venue should state the procedure before authors submit.
A check needs a correction path before it influences a decision. Authors need access to the evidence behind a flag and enough time to respond. Reviewers need to know whether a finding has been disputed or corrected. Editors need to retain responsibility for deciding how much weight to give it.
Hullman also notes that a shared summary can lead reviewers to repeat the same errors. A venue should preserve room for independent reading and compare the check with issues reviewers find on their own.
Confidentiality comes first. A venue handling unpublished manuscripts should establish permission, data retention, third-party processing, and access rules before sending any material to a service. Individual reviewers should follow the venue's policy too. Our guide on using AI to write a peer review explains that separate decision.
Try a claim check on a paper you can use
Agent Bayes provides a claim check for an individual researcher's project. It is an example of the evidence an author or reviewer can inspect, not a venue-wide review service.
The check compares claims in a paper with other papers in the project's libraries. It can show supporting and contradicting passages, identify a missing source or qualifier, and suggest a narrower wording. The paper being reviewed supplies context for its claims, but the check does not use that paper as evidence for its own claims.
The check can miss a passage or misinterpret a source, so read the cited pages before acting on a verdict. Its result depends on the papers in the project. Add relevant references first, and treat a missing source as a reason to expand the library, not as evidence against the manuscript.
For other stages of review, see our guides to checking your own paper, reviewing a paper, and confidential checks before submission. Our comparison of AI and human peer review examines a different question: which errors each can find.
For a paper you have permission to upload, add it and its references to a project. Select AI Peer Review Paper and Check support. The panel shows a credit estimate, then charges for the work used. The check requires Starter or higher. Follow the guide and read each cited passage.
New posts, straight to your inbox
No newsletter fluff, just an email when we publish something new.
Email me when a new post is published on the Agent Bayes blog. You can unsubscribe anytime. We'll first send a confirmation email, and we only use your details for this. See our Privacy Policy.