Who Checks the Citations?

Overview

Attorneys, judges, and pro se filers increasingly use AI to draft legal documents, yet these tools frequently fabricate citations. Despite predictions that newer models would hallucinate less or that court sanctions would deter negligent filers, we found over 1,000 filings containing fabricated citations — with this number growing year-over-year. This study evaluates whether AI-based systems can mitigate these errors by automatically detecting hallucinations. We propose a taxonomy of legal citation hallucinations grounded in actual court filings and introduce a dataset of 1,300 brief excerpts containing injected errors. Benchmarking five models in agentic and non-agentic settings reveals that while the latest iterations perform better — GPT-5 achieves 84.4% recall and a 55.0% F1 score in an agentic framework — all models struggle with subtle error categories. Agentic verification remains resource-intensive, with GPT-5 averaging 15.3 steps per excerpt. Furthermore, restricted information access limits the efficacy of even the best agents. This gap creates policy concerns, as it disadvantages both AI systems and litigants who lack subscriptions to commercial legal databases.

The Problem

Legal citations play a prominent role in U.S. legal practice. Attorneys must point to past judicial decisions and laws to make their case. Fabricating citations, or misrepresenting the content of those citations, is the same as pointing to made-up law to win the case — one judge called it:

"an abuse of the adversary system" that puts the integrity of the judicial process at risk. — Noland v. Land of the Free, 114 Cal.App.5th 426, 445 (2025)

The rapid adoption of large language models (LLMs) in the legal system has turned rare individual instances of fabrication into a systemic problem. Pro se litigants, trained attorneys, and even judges are using LLMs to generate briefs, motions, and other court filings. A prevailing argument has been that this problem is temporary — that hallucination rates would diminish as models improve and that high-profile sanctions would induce greater caution. Our findings challenge these assumptions.

Legal hallucination rates are not consistently decreasing across GPT model generations, while hallucinated citations in court filings grow steadily. Hallucination rate is the percentage of fabricated citations in all model-generated citations.

Judges repeatedly describe the resulting burden as an "enormous waste of judicial resources," and increasingly impose sanctions because "lesser sanctions have been insufficient to deter the conduct." Mid Cent. Operating Eng’rs Health & Welfare Fund v. HoosierVac LLC, 2025 WL 574234, at *3 (S.D.Ind. Feb. 21, 2025); Powhatan County Sch. Bd. v. Skinger, 2025 WL 1559593, at *10 (E.D.Va. June 2, 2025). Courts impose these sanctions while pro se litigants — who stand to benefit most from AI's ability to improve access to justice — are least equipped to detect hallucinated citations and lack access to commercial legal databases.

Through a controlled experiment querying eight generations of ChatGPT models on 92 legal drafting prompts, we find that hallucination rates are no longer consistently decreasing across model generations. Early GPT-4o models released in mid-2024 exhibit the lowest rates at 1.23%, substantially improving over GPT-3.5's ~25%. GPT-5.1 reverses this trend at 6.57%. Newer models also generate more citations per document, drawn from a broader and less canonical set of cases that are individually harder to verify.

The LePhantomCite Dataset

To assess the promise of AI in automatically checking legal filings, we introduce LePhantomCite (Legal Phantom Citation), a benchmarking dataset of legal brief excerpts augmented with injected hallucinations. It is accompanied by a taxonomy of citation hallucination types derived from failure modes observed in real court filings.

Hallucination Taxonomy

We propose five categories of legal citation hallucinations grounded in failure modes documented in actual court filings. Categories are not mutually exclusive, but only one hallucination type is introduced per citation in the dataset.

• Non-existent citation. The citation does not correspond to any real case — the reporter volume, abbreviation, or page number contain implausible values.
  See example

  Injected: Original 133 S. Ct. 1017 → Hallucinated 446 Cal. Rptr. 4th 183

  Real-world case: A pro se plaintiff cited "Graham v. Nyquist (1974)" but the citation points to no existing case (Sims v. Souily-Lefave, No. 2:24-cv-00831 (D. Nev. Apr. 15, 2025)).

• Case name mismatch. The reporter citation and case name refer to two different real cases. Created by replacing either the case name or the reporter citation with that of another existing case.
  See example

  Injected: Cinel v. Connick, 15 F.3d 1338 → Boone v. Vinson, 15 F.3d 1338

  Real-world case: An attorney cited "Tinch v. Video Indus. Servs., Inc., 2019 WL 1396975" but that Westlaw citation corresponds to Dolberry v. Jakob, 2019 WL 1396975 instead.

• Incorrect pincite. The citation refers to the correct case, but the cited page number does not support the quoted language or proposition. This is largely a structural barrier: page number information for many cases is only accessible through Westlaw and LexisNexis.
  See example

  Injected: 830 F.3d at 514 → 830 F.3d at 511

  Public repositories do not include official reporter pagination, which originates in commercial publisher formatting. This makes pincite verification infeasible without a commercial database subscription, explaining why all models fail on this category.

• Verbatim misquote. The exact quoted language does not appear in the cited case. Generated by replacing one or two words with semantically similar synonyms, preserving meaning so there is no content mismatch.
  See example

  Injected: "[T]he parties' dispute over arbitrability specifically falls within those carve-outs." → "[T]he parties' dispute over arbitrability specifically resides within those exclusions."

  Real-world case: In Harris v. Take-Two Interactive Software, a plaintiff cited Chambers v. NASCO, Inc. with a quotation that "courts have 'the inherent power to police themselves and to sanction bad-faith litigation conduct.'" This quote was not found in the cited decision.

• Content misrepresentation. A cited case exists but does not support the proposition for which it is cited. The hardest hallucination type to inject synthetically and to detect automatically.
  See example

  Injected: Original holding: "These rules cannot support a claim for retaliatory discharge." → Hallucinated: "Any private policies cannot support a claim for retaliatory discharge under Kansas law."

  Real-world case: In Jakes v. Youngblood, a defendant's attorney wrote that a Pennsylvania Superior Court case "emphasized that even where statements are embarrassing or upsetting, the Plaintiff must demonstrate their precise defamatory content and origin." The cited case contains no such opinion.

Benchmark Results

We evaluate five models in agentic and non-agentic settings using a custom harness with access to CourtListener search, local opinion retrieval, and open web search. We adapt BOED agent from Zheng et al., 2026 as our agentic framework. BOED agent maintains an explicit, language-based belief state that is updated after each action. We adopt this framework because case citation verification is inherently sequential and information-dependent: the agent must extract citations from the brief excerpts, decide on how to gather information and when it has gathered sufficient evidence to make a hallucination determination. We set a maximum of 30 steps per episode. To compare against a production agent harness, we additionally evaluate Claude Code with Opus 4.8, using the same prompt and a matching 30-turn budget.

Full precision / recall / F1 and per-type recall are in the leaderboard above.

• Claude Code achieves the best precision and F1. Running in its native harness with Opus 4.8, it reaches 76.1% precision and 68.8% F1, though its recall (62.8%) trails GPT-5, Gemini 2.5 Flash, and Qwen3.6-27B with the BOED agent. It uses tool calls efficiently: 15.4 per excerpt on average, with only 7.4% exact duplicates — although 21.5% of its CourtListener calls are invalid because it guesses tool names instead of discovering them first.
• Agentic retrieval raises recall for every model. GPT-5 improves recall by 26.2 percentage points with access to search tools (58.2% → 84.4%), the highest recall of any system.
• GPT-5 achieves the highest recall on every hallucination type. It catches 100% of non-existent citations and case name mismatches, 95.2% of verbatim misquotes, and 83.2% of content misrepresentations. Weaker models are considerably lower on content-based types, plateauing around 50%.
• Incorrect pincites remain intractable. No model reliably detects them: GPT-5 reaches 52.8% recall and Claude Code only 3.6%. This is a structural barrier — official reporter pagination is locked behind commercial legal databases, and 19.9% of opinions retrieved by agents contain no usable text or lack pagination.
• Precision is low for the BOED agents. High recall comes at the cost of false positives: GPT-5 reaches 40.8% precision, the best among BOED agents. Weaker models often treat absence from CourtListener as evidence of hallucination, inflating false positive rates.

Citation

@misc{liu2026checks,
  title         = {Who Checks the Citations? Benchmarking Legal Hallucination Detection},
  author        = {Liu, Patty and Stammbach, Dominik and Henderson, Peter},
  year          = {2026},
  eprint        = {2606.21155},
  archivePrefix = {arXiv},
  url           = {https://arxiv.org/abs/2606.21155}
}