The uncomfortable finding about AI hallucination in legal work is not that it happens. It is that it keeps happening in the systems specifically engineered to stop it — retrieval-grounded, domain-trained, sold by the incumbent legal research houses at enterprise prices. Any plan for using AI in document review that begins “we will use a tool that does not hallucinate” is not a plan. The tools that make that claim have been measured, and the claim did not hold.
What the evidence actually says
In the most-cited empirical work on the question, researchers at Stanford tested the commercial legal research assistants sold by LexisNexis and Thomson Reuters. Their finding, stated verbatim in the abstract:
— Magesh, Surani, Dahl, Suzgun, Manning & Ho, Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools, published in the Journal of Empirical Legal Studies (2025).[1]
Two details matter more than the headline number.
The first is what those systems were. These are not general chatbots. They are retrieval-augmented products built on curated, proprietary legal databases — the exact architecture the industry proposes as the answer to hallucination. Grounding reduced the error rate. It did not eliminate it. The paper's own conclusion is that providers' claims of hallucination-free output are overstated.
The second is scope, and it cuts against overusing this study. The research measured legal research — open-ended questions answered against a corpus of case law. That is not the same task as reading a specific contract that sits in front of you and reporting what it says. The retrieval step, which is where a large share of grounding errors originate, is far more constrained when the document set is fixed and known. Anyone citing this study as a direct measurement of contract-review accuracy is stretching it, and you should discount them accordingly.
Where models actually invent, specifically
“Hallucination” is a single word covering at least five failure modes with different detection costs. Treating them as one thing is why so many verification processes catch the easy ones and miss the expensive ones.
| Failure mode | What it looks like | How hard to catch |
|---|---|---|
| Fabricated source | A cited clause, section number, case or exhibit that does not exist | Easy — the reference resolves to nothing |
| Misgrounded citation | A real clause cited for a proposition it does not support | Hard — the link works, so a spot-check passes |
| False synthesis | Two real provisions combined into a conclusion neither supports | Hard — every component is verifiable, the join is not |
| Confident omission | A material provision simply not mentioned, with no signal it was missed | Hardest — there is nothing on the page to check |
| Entity drift | Obligation attributed to the wrong party, subsidiary, or counterparty | Moderate — needs cross-document reading to spot |
Notice the pattern. The failure modes that a reviewer's instinct catches — invented case names, references that resolve to nothing — are the cheap ones. The ones that survive review are the ones where everything checks out individually. A misgrounded citation passes a click-the-link test. A false synthesis passes a verify-the-quotes test. And confident omission defeats verification entirely, because verification operates on what the model produced, and the failure is something it did not produce.
That asymmetry has a direct consequence for how you design controls: a verification process built only around checking what the AI said will systematically miss what the AI never said. Coverage has to be handled structurally, not by review.
Why document review carries a different risk shape
In legal research, a hallucination is usually loud. You cite a case, opposing counsel cannot find it, and the failure surfaces immediately and publicly — the reason the profession learned this lesson through sanctions rulings rather than white papers.
Diligence is quieter and, for that reason, worse. A missed change-of-control provision does not announce itself at the deal table. It surfaces eighteen months later, when the counterparty exercises it, and by then the transaction has closed, the purchase price is spent, and the leverage that would have let you negotiate an indemnity is gone. The error is discovered by the person harmed by it, not by the person who made it.
This is the argument for treating diligence AI to a stricter standard than research AI, not a looser one. The feedback loop that eventually corrects public-facing errors does not exist here.
Detection that survives contact with a live deal
Verification protocols tend to be written in calm conditions and abandoned in week three of a deal. The ones that hold share a property: they cost near-zero marginal effort per finding, because anything that costs real effort per finding gets skipped exactly when volume is highest.
1. Require the quote, not the summary
Every assertion should carry the source text it rests on — document, page, and the actual sentence. This is the single highest-leverage control available, and it is cheap to enforce because it is a property of the output format rather than of reviewer discipline. An assertion with an attached quote can be checked in seconds. A paragraph of confident prose cannot be checked at all without re-reading the source, which defeats the purpose of the tool.
2. Treat quote-free findings as a distinct, lower class
If a system cannot produce supporting text for a claim, that claim should be visibly demoted rather than silently mixed in with grounded ones. The failure mode here is presentational: a well-supported finding and an ungrounded inference rendered in identical type look equally authoritative, and reviewers are not able to tell them apart by reading.
3. Verify negatives separately
“No change-of-control provision found” and “this category was not examined” are wildly different statements that look identical in a clean report. Absence of a finding must be distinguishable from absence of a search. Systems that assign a category an explicit floor when nothing was found — rather than reporting a comfortable zero — make this visible instead of hiding it.
4. Sample the boring middle
Reviewers naturally verify the alarming findings. Errors concentrate in the unalarming ones, because nobody looks. A fixed random sample of low-severity output, checked every deal, is the only way to measure your actual error rate rather than assuming it.
The structural fix: make the output checkable by construction
Every control above is a workaround for the same root problem — that free-text AI output is expensive to verify. The more durable response is to change the shape of the output so verification is cheap by default.
This is the practical case for putting a scoring stage between document intake and human review, rather than asking a model to write you a narrative. A score against a defined category, carrying the source quote that produced it and a threshold that determines whether it blocks, has three properties a summary does not:
- It is falsifiable. A number attached to a quote can be disagreed with precisely. “This scored 82 on the strength of this indemnity clause” invites a reviewer to say no, that clause is standard, score it 30. Narrative prose invites nodding.
- It is consistent across documents. The same category, the same rubric, the same threshold, applied to document 900 as to document 3 — which is precisely where human reviewers degrade, not through incompetence but through fatigue.
- It makes coverage auditable. Every document has a score in every category, so the confident-omission failure mode becomes visible as a low score with no supporting quote, rather than as silence.
None of this makes the underlying model more accurate. That is the point worth being clear about: scoring does not reduce the hallucination rate, it reduces the cost of catching hallucinations. Those are different claims, and only the second one is defensible.
What the professional rules now require
This stopped being purely a quality question in July 2024, when the ABA Standing Committee on Ethics and Professional Responsibility issued Formal Opinion 512, its first formal guidance on generative AI. The opinion holds that lawyers and firms using these tools must “fully consider their applicable ethical obligations,” naming duties of competence, confidentiality, communication with clients, candor toward tribunals, supervisory responsibility, and charging reasonable fees consistent with time actually spent.[2]
Read against the failure modes above, the competence duty is the operative one. It is not satisfied by choosing a reputable vendor. It attaches to the output that leaves your desk — which means the verification path is not an internal quality preference, it is the thing that makes the use of the tool defensible.
Bottom line
Assume your tools hallucinate, because the measured evidence says the best-resourced ones in the adjacent domain do. Then stop trying to solve it at the model layer, where you have no leverage, and solve it at the output layer, where you have all of it: demand quotes, separate grounded findings from ungrounded ones, make absence of a finding distinguishable from absence of a search, and sample the parts nobody wants to check.
A tool that is occasionally wrong but always shows you where it got its answer is safe to build a process on. A tool that is usually right but unfalsifiable is not — and the second one demos far better than the first.
Sources
- Magesh, V., Surani, F., Dahl, M., Suzgun, M., Manning, C. D., & Ho, D. E. Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools. arXiv:2405.20362; published in the Journal of Empirical Legal Studies (2025). arxiv.org/abs/2405.20362
- American Bar Association Standing Committee on Ethics and Professional Responsibility, Formal Opinion 512: Generative Artificial Intelligence Tools, 29 July 2024. americanbar.org
We cite only sources we have retrieved and read. Where a figure could not be verified, we changed the figure rather than the citation — see our methodology.