Definition
Ground truth is target data treated as correct for a specific evaluation case. It could be an extracted field value, a document label, or the facts an answer must contain. It may come from human review, a verified record, or a deterministic calculation. Record how each target was established.
“Ground truth” describes the target’s role in a test. A label can still be disputed, and a correct policy can later change. The evaluator also needs a method for comparing an output with the target.
Simple example
A support assistant answers a question about replacing a damaged delivery. A reviewer checks the current policy and records that replacement is available within 14 days of delivery, along with the policy version used. That reviewed rule is the ground truth for the test case.
An answer claiming a 30-day window fails the check. “Request a replacement within two weeks of delivery” can pass even though it does not repeat the reference wording.
Why it matters
Ground truth lets a team check whether a prompt or model change still handles known cases correctly. If an evaluation needs a known correct answer but has no reliable target, its score may reward similarity to an arbitrary sample answer or rely on an evaluator’s unsupported guess.
Recording the source and version also makes a failure easier to investigate. The team can tell whether the system missed the rule or the evaluation case itself needs correction.
One important nuance
Ground truth can be wrong or go stale. Reviewers may disagree about a label, and open-ended questions may have several valid answers. When the replacement policy changes, revise the affected target and keep the old dataset version with earlier results. Otherwise, a score difference across runs may reflect a changed answer key rather than a changed system.