Updated 13 September 2026: this article now presents a general method. Examples are illustrative, not customer results.
An AI reviewer can invent an authority for its answer, even when its instructions say to cite evidence. I therefore treat a rule identifier as a claim to verify, not as proof that a check was performed.
Consider a fictional example: an answer says that a missing check is permitted by a rule that does not appear in the supplied ruleset. The limitation may be real, but the invented reference does not justify the decision.
Why this is worse than inventing a fact
A rule citation can make an answer look procedurally complete. If the referenced rule does not exist, a reviewer cannot follow the claimed reasoning back to an authoritative source.
I separate three questions: does the source exist, does the cited passage say what the answer claims, and does that passage support the conclusion? Passing the first check does not settle the other two.
The fix is a rule about quotability, not a plea for honesty
- Use an agreed source. Record the ruleset and version against which the answer is checked.
- Verify identifiers. A claimed reference must be found in that source.
- Describe limitations plainly. Missing evidence can be reported without inventing a procedural justification.
- State coverage. Identify which checks were performed and which remain unresolved.
The workflow should make an unresolved answer usable. A person can decide what evidence or clarification is needed next.
Test the limits of mechanical checks
Code can fail too. A mechanical check is useful only within its tested coverage. It can reject an identifier absent from a supplied list, but it cannot establish that the list is complete or that a judgment is correct.
The public fictional source checker illustrates this distinction. It finds or rejects a quotation on a supplied page. It does not validate the conclusion drawn from that quotation or demonstrate general AI accuracy.
I test correct references, invented references, wrong versions, missing evidence and ambiguous cases. Reviewers still decide whether a finding stands.
What to take from this
- Treat citations as claims to verify.
- Give the system a way to report missing information without filling the gap.
- Use mechanical checks for properties that can be tested explicitly.
- Measure missed errors and unnecessary rejections as well as apparent successes.
Questions people ask about fabricated citations
Does a strict prompt prevent invented references? It is not a guarantee. Check the output against the agreed source.
Does a correct citation prove the answer? No. A real passage can be quoted accurately and still used to support an incorrect conclusion.
What can code verify? Defined properties such as whether a supplied identifier or quotation exists in a given source. The coverage and failure cases need testing.