Definition
Factual consistency is the degree to which claims in an output agree with the reference information used for evaluation. A contradiction, unsupported alteration, or incorrect relationship between stated facts reduces consistency even when the output is fluent and relevant.
The reference may be a source document, structured record, or set of statements. The evaluation question is narrow: does the output preserve the facts represented there?
Simple example
A source incident report states that a deployment began at 14:00, failed during database migration, and was rolled back at 14:18. A summary that says the deployment completed at 14:18 is factually inconsistent with that report. A summary that omits the start time can still be consistent if it does not make a conflicting claim.
Reviewers can evaluate each generated claim against the relevant reference statement instead of assigning one general quality score.
Why it matters
Summarization, extraction, and transformation systems can change facts while producing plausible text. A factual-consistency check targets those changes separately from writing quality, relevance, or completeness.
Separate checks show whether a response contradicted available information, left out required content, or was evaluated against a poor reference.
One important nuance
Factual consistency and faithfulness overlap, and some evaluations use the terms interchangeably. When an evaluation distinguishes them, faithfulness usually asks whether claims are supported by designated source evidence. Factual consistency asks whether the claims agree with the reference used for evaluation. Neither establishes real-world truth if that evidence or reference is outdated, incomplete, or wrong.