Run production evaluations outside the user response path
Hand production samples to a separate evaluation worker instead of making users wait for an evaluator model.
Csharp
Attach build identity to operational telemetry
Stamp telemetry with the immutable build that produced it so incidents can be compared across releases and rolling deployments.
Use FrozenDictionary for stable, read-heavy lookup data
Build stable lookup tables once with FrozenDictionary, reuse them across reads, and measure whether lookup savings justify the construction cost.
Return Retry-After when a client should back off
Tell API clients when another attempt may succeed instead of leaving them to guess a retry delay.
Version prompts, models, datasets, and evaluation results together
Keep every evaluation result tied to the exact prompt, model identities, dataset, evaluator, and configuration that produced it.
Choose histogram boundaries from the service objective
Put the latency objective into the histogram boundaries so compliance can be calculated without interpolating between buckets.
Provider Independence from Day One
A C# implementation for keeping provider coupling at explicit boundaries while making model capabilities and provider-specific behavior visible.
Use concurrency tokens to detect lost updates
Make stale writes fail explicitly instead of silently overwriting newer database state.
Primary constructor parameters are not automatically properties
A primary constructor parameter stays a parameter. Declare a property or field when the value should be an explicit member of the type.
Separate ingestion health from retrieval readiness
A failed ingestion run should not stop retrieval while an accepted index remains queryable and eligible to serve.