Run production evaluations outside the user response path
Hand production samples to a separate evaluation worker instead of making users wait for an evaluator model.
Llm-Evaluation
Version prompts, models, datasets, and evaluation results together
Keep every evaluation result tied to the exact prompt, model identities, dataset, evaluator, and configuration that produced it.
Testing Microsoft Agent Framework Applications
How to test Microsoft Agent Framework applications with fake model clients, tool contract tests, structured output tests, routing tests, workflow tests, and eval-style regression checks.
Use evals before changing prompts
Treat prompt changes like code changes: measure the behavior before deciding whether the edit helped.
Stop Guessing – Use Golden Datasets for Prompt Evals
Use a small golden dataset to catch prompt regressions, compare changes against a baseline, and validate model updates before users do.
RAG Is a Data Problem Before It’s a Prompt Problem
Why stale documents, weak chunking, and thin metadata usually break RAG before prompt tuning does.
Eval-first: Why “It Worked Once” Is Not a Sign of Quality
Why eval-first matters for LLM apps and how to use datasets, scoring rubrics, and CI quality gates to catch regressions early.