Definition
Evaluation-driven development uses repeatable evaluations throughout the development of an AI system. Engineers define the behavior they need, run cases against the current system, make a change, and compare the results before deciding what to do next. The results guide changes to prompts, retrieval, models, tools, or application code.
The term describes a development workflow, not a particular metric or evaluation tool. A final release check can use the same tests, but the defining feature is that results influence decisions while the system is being built.
Simple example
A support assistant sometimes answers a refund question using the policy for the wrong country. Engineers add reviewed cases for several countries, recording the applicable policy and the facts each answer must get right. A run against the current system gives them a baseline.
They add a country filter to retrieval and rerun the cases. Wrong-country answers decrease, but a few questions now return no policy at all. Those failures need investigation before the change is accepted. The results point to the next piece of work instead of merely giving the release a pass or fail.
Why it matters
An AI system can improve on the example an engineer just inspected while getting worse elsewhere. Comparing runs across relevant cases can reveal regressions that a single example would miss. A team can turn a reported failure into a case to rerun after a fix.
One important nuance
Repeatedly tuning against one fixed test set can make the system good at those cases without improving real use. Keep some cases held out for independent checks. If the team adjusts the system to improve those results, the cases lose their independence. Add cases from observed failures, and review the scoring when the product’s required behavior changes.