How to Check Whether AI Citations and Numbers Are Real

Verify that each AI citation exists, locate the exact supporting passage, recalculate important numbers, and check the surrounding context before use.

A working link does not prove that the source supports the AI’s sentence. A calculated percentage does not prove that the denominator, period, unit, or population is correct. Check four layers separately: whether the source exists, whether it supports the claim, whether you can reproduce the number, and whether the time and scope match your use.

Imagine an AI summary that says, “According to a 2024 report, teams using AI increased productivity by 40%.” It includes a PDF that opens successfully. That feels reassuring.

Inside the report, however, the 40% may describe the time required for one test, involve only a particular job, or come from a calculation the AI assembled from two unrelated tables. A real link passes only the first check.

NIST’s Generative AI Profile recommends reviewing and verifying sources and citations in generative AI outputs. It also warns against generalizing model capabilities from narrow or unsystematic tests. NIST: Generative Artificial Intelligence Profile

NIST supports source verification, documented limitations, and evaluation in context. The four-layer sequence below—existence, support, reproducibility, and scope—is my method for general readers. It is not an official NIST personal checklist.

First: confirm that the original source exists

Check the source’s identity before reading the claim:

Do the title, author, and publisher match?
Does the link point to the original publisher rather than a repost or search snippet?
Can you identify the publication date and version?
Does the quoted page, section, or table actually exist?

If the AI writes “a study found” without enough information to locate it, ask for a title, author, or link—but do not treat the generated citation as verified. Search the official publisher, paper database, or organization using those details.

When the original source cannot be found, record “not located,” not “probably true.” Do not present the statement as confirmed in an external article, presentation, or consequential decision.

Second: check whether the source supports the whole sentence

Break the AI sentence into separate claims:

> teams using AI / productivity increased / 40% / applies to general work

Then ask:

Were the subjects actually teams, or individual participants?
Did the study measure overall productivity, or time on one task?
Was 40% an average, maximum, relative difference, or subgroup result?
Can the result reasonably be applied to your situation?

A source may contain the number “40%” without supporting the combined sentence.

AI claim	Source location	What the source says	Support
AI increased team productivity by 40%	Table 2	Specific participants completed one writing task in 40% less time	Partial; it does not establish productivity for all teams

Avoid a simple true/false label. Record which part is supported and which part exceeds the evidence, so you know how to revise the sentence.

Third: reproduce the calculation

For any number, identify the numerator, denominator, unit, and period.

“Errors fell 50%” could mean two errors became one. “This quarter increased 20%” may mix people with transactions. An average processing time may exclude the slowest cases.

When the original values are available, calculate the change yourself:

Rate of change = (new value - old value) / old value

If the old value is 10 and the new value is 8, the change is -20%, or a 20% decrease. Percentage and percentage points are different: a rate moving from 40% to 50% rises by 10 percentage points, while its relative increase is 25%.

If the AI rounded, excluded data, or combined fields, ask it to show the values and formula. Then reproduce the calculation independently. “I checked again and it is correct” is not independent verification.

Fourth: match the date, population, and scope

Even a correctly quoted result may not fit your use:

the source refers to an older product version;
the study involved software engineers, but the article generalizes to all knowledge workers;
the result came from a controlled test, but the claim describes production work;
the finding applies to one country, plan, or dataset;
a correlation is rewritten as “A caused B.”

The NIST AI RMF Core calls for measurement and verification methods that reflect context and risk, with limitations and uncertainty documented. NIST: AI RMF Core For everyday work, the practical lesson is simple: the farther the evidence is from your situation, the narrower your conclusion should be.

When evidence fails, remove the attractive sentence

There are four reasonable outcomes:

Keep: The original source supports the claim, the number is reproducible, and the scope matches.
Rewrite: The evidence supports a smaller or more conditional statement.
Mark for review: You lack the original data or cannot reproduce the value, so the claim stays out of external use.
Delete: The source does not exist, does not support the claim, or the sentence is unnecessary.

Good verification is not a search for a defense of every AI-written sentence. It includes being willing to remove claims that the evidence cannot carry.

Practice: review an intentionally overstated summary

A test asked 20 participants to complete one summarization task.
The tool group took 12 minutes on average; the comparison group took 15 minutes.
The AI summary says: “The study proves AI increases overall productivity
for all knowledge workers by 25%.”

Complete this record:

Citation and number review

Claim:
Does the source exist?
Population and task actually studied:
Original values:
My calculation:
Where the AI sentence exceeds the evidence:
Action: keep / rewrite / mark for review / delete
Evidence-supported rewrite:

Completion time fell from 15 minutes to 12 minutes, a 20% reduction. If the amount and quality of work were identical, output per minute would rise by 25%, but that requires an explicit productivity definition and additional assumptions. Neither calculation proves an effect on all knowledge workers or overall productivity.

After this exercise, you can review claims that trace back to a source. The next lesson handles statements that are not directly verifiable facts—such as inferences, recommendations, and decisions—which require different tests.