What Happens When Your AI Feature Fails?

The first design pass for an AI feature should describe how it fails.

That can feel backwards when the feature does not work yet. But a timeout is already part of the design. So are a retrieval miss, malformed output, and a write that may have completed after the caller gave up waiting.

Start with one user-visible flow. At every boundary, ask what can go wrong, what may already have happened, what the application can still promise, and how anyone will know which case occurred.

Keep the answers in a failure-mode table and use it to shape the API contract, control flow, telemetry, and tests. Writing that table after the first incident is rather late.

The happy path hides product decisions

Consider a support feature that prepares a reply and optionally saves it as a draft:

resolve ticket and evaluate access
    -> retrieve authorized ticket content and policy context
    -> call the model
    -> validate the generated reply
    -> save the draft when requested
    -> return the result

The sequence is easy to understand. It also leaves almost every difficult question unanswered.

What does the user see when retrieval returns no policy documents? Does the model get a chance to answer anyway? If output validation fails, can the application repair it? If saving times out, is the draft absent, stored, or still being processed? Can a retry create a second draft? Which failure should wake an operator?

Those are part of the feature contract. If the team postpones them until implementation, the answers tend to become whatever falls out of an exception handler or SDK default.

I would rather make the awkward cases visible first. Once those decisions are written down, implementing the successful case tends to be the easy part.

Begin with the promise and the unacceptable outcomes

Before listing infrastructure failures, write down what the user believes the operation does.

For the support feature, the promise might be:

Prepare a reply from the ticket and approved support policy. Save it as a draft only when the user asks.

Now write the outcomes the system must not present as success:

• a reply based on another tenant’s data
• a policy claim with no approved source
• a valid-looking reply that failed the output contract
• a “saved” result when persistence was never confirmed
• a duplicate draft created by retrying an ambiguous write

This short list changes the design. Missing retrieval context cannot silently become a normal model request. A model response cannot become a persisted draft before it passes the output contract. Save success requires a confirmed write, not the absence of an exception.

The unacceptable outcomes matter more than a generic goal such as “handle errors gracefully.” They state which properties the application must preserve when part of the flow fails.

Strictly speaking, not every row in this exercise is a system failure. An authorization denial can be correct behavior. NoEvidence can be a valid domain outcome. I include those adverse outcomes because they can still prevent the feature from keeping its promise, and the application needs an explicit response for them.

Start from the interaction map

If you already mapped the dependencies around the model, use that interaction map as the input. Otherwise, sketch the important calls and state changes for this one flow. Another inventory of Azure services, SDK clients, and databases will not tell you how the feature should behave.

A service can fail differently in different interactions. Reading policy context may permit a reduced result. Authorizing access does not. A database read and a draft write may use the same database but have different consequences when their outcomes are unknown.

Take one interaction at a time and ask:

1. How can this interaction be slow, unavailable, wrong, stale, unauthorized, duplicated, or ambiguous?
2. What is the effect on the user-visible operation?
3. Could data or external state already have changed?
4. What response preserves the feature’s promise?
5. Which signal distinguishes this case in production?
6. How will a test force it to happen?

Use the category list to jog your memory; stop when more rows would describe the same effect and response. A model call can return valid JSON with an unsupported claim. A write can succeed while its acknowledgement is lost. Caller cancellation can arrive after work started. These cases are more useful than another row that simply says “dependency unavailable”.

Build a table that can change the code

For the support flow, a first pass could look like this:

If the cells do not affect implementation, the table is busywork. “Log the error and retry” leaves open whether retrying is safe, what the caller receives, and when the operation stops.

An external effect and a local audit record do not share a transaction boundary. Record durable intent before the effect. Afterward, reconcile and record the final outcome. Depending on the operation, that recovery path may also need an idempotency key or compensation.

Do not try to enumerate every exception type. Group failures when they have the same effect and response. Split them when the system must behave differently.

Separate failure, effect, and response

Teams often jump from a technical symptom to a resilience mechanism:

timeout -> retry
invalid output -> retry
dependency unavailable -> fallback

That skips the decision that matters: what did the failure do to the operation?

A timeout on an idempotent policy read is not the same as a timeout after a draft write was submitted. Both may present as a timeout at the application boundary. The read has no side effect and might be attempted again within the remaining budget. The write has an ambiguous outcome. Retrying it with a new identity may create a duplicate.

Choose the response from the effect and the known state. The exception name is only one input.

For each row, I use one of a small set of response shapes:

• stop and return a specific application outcome
• continue with an explicitly reduced capability
• make another bounded attempt when the operation is safe and time remains
• move unresolved work to a durable recovery path
• require human review before a consequential action
• fail closed because authorization, integrity, or policy cannot be proven

The exact set belongs to the application. Callers should receive application outcomes without having to interpret exception text.

Put the outcomes in the application contract

Once the failure table stabilizes, encode the outcomes that the endpoint or UI needs to handle.

public enum ReplyOutcome
{
    Prepared,
    NoEvidence,
    InvalidGeneration,
    TimedOut,
    TemporarilyUnavailable
}

public enum DraftSaveOutcome
{
    NotRequested,
    Saved,
    RejectedByPolicy,
    Failed,
    Unconfirmed
}

public sealed record PrepareReplyResult(
    ReplyOutcome ReplyOutcome,
    string? Reply,
    DraftSaveOutcome SaveOutcome,
    Guid? SaveOperationId);

RejectedByPolicy means the application deliberately refused the save before persistence. Failed means the application knows that persistence did not commit. Unconfirmed means the write may have committed, but the application has not verified the result.

Allocate the operation ID before the write:

Guid operationId = Guid.NewGuid();

var command = new SaveDraftCommand(
    OperationId: operationId,
    TicketId: ticketId,
    Reply: reply);

DraftSaveResult saveResult = await draftStore.SaveAsync(
    command,
    cancellationToken);

The store must persist OperationId with the draft or an operation record and enforce uniqueness. Reusing the ID for a different command must fail. Persist enough command identity, such as the ticket ID and a canonical command fingerprint, to detect conflicting reuse. If the response disappears after commit, the application queries that identity, and any safe retry reuses it. Otherwise, the ID gives a worker nothing authoritative to reconcile.

The compact result type still permits invalid combinations. Production code may use factory methods or a result hierarchy to prevent them. Even so, it makes two decisions explicit: reply generation and draft persistence have separate outcomes, and an unconfirmed save is not reported as either success or failure.

The endpoint can now map those outcomes deliberately. The UI can say that a reply was prepared but its save status is still being checked. A background worker can reconcile the same operation ID. Telemetry can record stable outcome names instead of provider-specific exception messages.

Prioritize without pretending the numbers are precise

A full failure-mode analysis can grow quickly. Do not give every row equal attention.

In a review, I start with three questions:

• How much harm can this cause to users, data, security, or an operational promise?
• How plausible is the case in this design and environment?
• How likely are we to detect it before the user or an operator has to report it?

I prefer simple priority labels such as critical, high, normal, and low over multiplying guessed scores into a number that looks scientific. Unknown write outcomes, cross-tenant data exposure, silent use of stale policy, and failures that look like success deserve attention before a clean provider error that the application already exposes honestly.

Keep the treatment decision separate: mitigate, accept, or defer. A high-priority risk can still be accepted when the team understands the consequence and decides that mitigation is not justified. “We do not support draft recovery in the first release” can be a legitimate decision if the UI never claims an unconfirmed write succeeded and the consequence is acceptable. An undocumented gap is not the same thing as an accepted risk.

Turn every important row into a forced failure

For every important row, the team should be able to force or simulate the relevant condition and effect. Some provider and infrastructure failures cannot be reproduced exactly, but their application-visible behavior usually can.

You do not need a production-scale chaos platform for the first pass. Use fake clients and controlled test doubles to return no context, delay a model call, produce malformed output, reject authorization, or lose a write acknowledgement after commit.

For each high-priority row, verify four things:

1. The application returns the intended outcome.
2. It preserves the required data and authorization properties.
3. It emits enough context to identify the interaction and failure category without leaking sensitive content.
4. Any retry or recovery path preserves side-effect safety.

Keep at least the deterministic cases in the normal test suite. Run slower dependency and recovery drills on a schedule that the team will actually maintain.

The same rows guide the code and tests. During an incident, they also tell the operator which behavior was intentional.

Give retries and runtime budgets their own pass

The analysis should identify where another attempt may be useful. Decide retry behavior holistically across the operation afterward. That does not mean retrying the whole workflow.

Whether an attempt is safe depends on idempotency, the failure class, remaining time, provider throttling, and the scope of any side effect. A retry can be a valid response to one row and make the next row much worse.

A five-second model timeout tells you little on its own. It has to fit inside the request budget for retrieval, model calls, validation, tools, persistence, and any synchronous recovery attempts. Durable recovery that outlives the request needs its own deadline or operational objective.

During the failure pass, note which rows need a retry or budget decision. Resist inventing a local retry count or timeout just to fill the cell.

When to use this approach

Use a failure-first pass when an AI feature reads protected data, relies on retrieval, makes authoritative claims, invokes tools, changes state, or creates a meaningful promise about latency and availability. It is also useful before changing a prompt, model, or provider when that change can alter output contracts or runtime behavior.

When a lighter pass is enough

A disposable local experiment with synthetic data and no side effects does not need a workshop or a large register. Write down the few cases that could invalidate the experiment, keep the error visible, and move on.

Keep the method proportional to the feature. A small experiment needs a short list, not a reliability program.

Practical takeaway

Choose one important AI flow and spend 45 minutes on its unhappy paths before adding more happy-path code.

Write the user promise and the outcomes that must never appear as success. Walk each interaction, record the effect of plausible failures, and assign an application response, a diagnostic signal, and a way to force the case in a test.

If a row has no defined response or cannot be tested, it is unfinished design work.

Related reading

• Calling a Model Is Easy. Running an AI System Is Not
• The Model Is Only One Dependency. Map the Rest.
• When an AI Request Should Become a Background Job

Sources

• Microsoft Learn: Architecture strategies for performing failure mode analysis
• Microsoft Learn: Transient fault handling
• Microsoft Learn: Connection resiliency in Entity Framework Core
• Microsoft Learn: Architecture strategies for designing a reliability testing strategy