Why does ChatGPT make up references?

It helps to see what the model is doing as pattern completion rather than lookup. A citation has a highly regular shape: two or three surnames, a year in parentheses, a title with the right technical vocabulary, a journal whose name sounds like the field, a plausible volume and page range. A model trained on millions of reference lists reproduces that shape without difficulty. Producing a reference that is also true requires something the generation step does not do, which is to consult a record of what has actually been published.

This is why fabricated references are so convincing. They are not random. The invented authors are usually real researchers who publish in that area. The invented journal is usually a real journal. The invented title is usually the paper you would want to exist. Occasionally the model produces a real title attached to the wrong authors, or a real paper with a corrupted year and volume, which is harder to catch than a complete invention because the first search result looks like a match.

The pattern also explains when the problem is worst. Recalling literature from memory is the worst case. Asking a model to answer from documents you have supplied reduces it substantially, because the reference can be tied to a text the system actually has. It does not remove the need to check, and no architecture makes verification optional.

The published work supports that direction while disagreeing about how far it goes. Lewis and colleagues paired a language model with a retrievable Wikipedia index and found the resulting system generated more factual, specific, and diverse text than a parametric-only BART baseline, reaching state-of-the-art results on Natural Questions, WebQuestions, and CuratedTrec and coming within 4.3 percent of specialized pipeline models on FEVER fact verification (Lewis et al., 2020, pp. 2, 9). Xu and colleagues argue a hard limit in the other direction, that hallucination cannot be eliminated at all, because a language model cannot learn every computable function and will therefore always be inconsistent with some ground truth (Xu et al., 2024, p. 1). Their claim is about complete elimination, not about every output being wrong.

How common are fabricated citations in AI output?

It depends entirely on the model and the task, so treat any single percentage with suspicion. In one 2023 study of short papers generated with references, 55 percent of GPT-3.5 citations and 18 percent of GPT-4 citations did not exist, and many of the real ones carried substantive errors.

That study, by Walters and Wilder in Scientific Reports, is worth reading in full because it separates two failure modes that get collapsed together. A fabricated citation points at nothing. An erroneous citation points at a real paper but gets the authors, year, or page range wrong, which is the kind of mistake that survives a casual check and then propagates into your reference list.

The breakdown by source type tells you what to check first. Across 84 generated reviews on 42 topics containing 636 references, GPT-3.5 fabricated 73 percent of the journal articles it cited, 70 percent of the book chapters, 50 percent of the websites, and 23 percent of the books (Walters and Wilder, 2023, p. 4). GPT-4 cut article fabrication to 18 percent, books to 8 percent, and websites to 10 percent, but chapter fabrication stayed at 70 percent in both models. A cited chapter in an edited volume is the reference most likely to be invented, whichever model produced it.

Among references that did point at real work, the errors sit in the fields you are least likely to read closely. Incorrect volume, issue, or page numbers appeared in 34 percent of GPT-3.5 article and chapter citations and 13 percent of GPT-4 citations, and incorrect publication dates in 22 percent and 16 percent (Walters and Wilder, 2023, p. 4). The same study found five generated reviews written up as empirical studies with invented methods and results, and one GPT-4 response reporting fabricated correlation coefficients, regression coefficients, and p values (Walters and Wilder, 2023, p. 6). Fabrication is not confined to the bibliography.

Other published measurements from the same period range from roughly a quarter to more than two thirds fabricated, depending on the discipline, the prompt, and the model. The spread is the finding. There is no stable rate to quote, which is exactly why "the newer models are better now" is not a verification strategy.

How do I check whether a citation is real?

Search the exact title in Crossref, PubMed, Scopus, or your library catalog, not in the chat window. If the title returns nothing, the reference is fabricated. If it returns a real paper, open it and confirm the authors, year, journal, and page range match, then read the passage that supposedly supports the claim.

Asking the model whether its own citation is real does not work, because the same generation process produces the confirmation. Neither does a DOI resolving, on its own, prove much, since a fabricated reference sometimes carries a DOI belonging to a different paper.

Walters and Wilder tested that request directly and report that ChatGPT frequently stood by incorrect claims when asked to verify them, which follows from their description of it as a language-processing tool rather than an information-processing one, imitating the textual patterns of scholarship without reproducing the underlying content (Walters and Wilder, 2023, p. 1). A confident second opinion from the same system is the same first answer restated.

The last step is the one people skip and the one that matters most. A reference can exist, be formatted correctly, and still not support the sentence attached to it. Verification is finished when you have read the relevant passage, not when the search returned a hit.