Table of Contents

Provider independence starts before the second provider arrives. Draw the provider boundary when the first integration is built. Alternate implementations can wait until a real need appears.

IChatClient gives .NET applications a common API for messages, responses, streaming, tools, and structured output. That removes a lot of SDK coupling. The remaining differences still need a place in the design. JSON Schema support, tool behavior, context limits, model names, deployment names, and provider-only options do not disappear because two clients implement the same interface.

Provider wiring stays at the composition boundary. A dedicated factory selects and constructs the Azure OpenAI or Ollama client. The ticket-routing feature uses one C# service for either backend. A small capability declaration selects native JSON Schema or prompt-guided schema instructions. A qualification check confirms that the configured structured-output path works before the backend is promoted.

That is the level of independence I trust: one application path with the differences exposed and backed by tests against each configured backend.

There is a complete example available as a .NET 10 file-based app. It runs offline by default and includes configuration examples for Ollama and Azure OpenAI.

Define the portable behavior first

Start with the feature, not a list of providers.

This example routes a support ticket to one of three queues. The model call needs:

  • system and user messages
  • a typed JSON result
  • cancellation
  • no tools, images, or provider-managed conversation state

Those requirements fit IChatClient. The application result does not need to expose chat messages or SDK response types:

public enum TicketQueue
{
    Billing,
    TechnicalSupport,
    AccountAccess
}

public sealed record TicketRoutingDecision(
    TicketQueue Queue,
    bool RequiresHumanReview,
    string Reason);

This is the portability contract for the first version. If the feature later requires hosted file search or another provider-only API, the contract has changed. Pretending otherwise would only move the coupling into a loosely typed options dictionary.

Install the provider integrations

The sample uses Azure OpenAI in a hosted environment and Ollama for a local backend:

dotnet add package Microsoft.Extensions.AI
dotnet add package Microsoft.Extensions.AI.OpenAI
dotnet add package Azure.AI.OpenAI
dotnet add package Azure.Identity
dotnet add package OllamaSharp

I compiled the code in this article with .NET 10, Microsoft.Extensions.AI 10.9.0, Microsoft.Extensions.AI.OpenAI 10.9.0, Azure.AI.OpenAI 2.1.0, Azure.Identity 1.21.0, and OllamaSharp 5.4.30. Check the current package documentation when using newer versions.

The dependency direction stays small:

TicketRouter
    -> SupportModelClient
        -> IChatClient

composition root
    -> SupportModelClientFactory
        -> AzureOpenAIClient or OllamaApiClient

TicketRouter has no reference to either provider SDK.

Keep model and deployment configuration separate

Azure OpenAI addresses a deployment. Ollama addresses a model available at its endpoint. The application should not use either value as its stable service name.

Use one options type for the selected support-routing backend:

using System.ComponentModel.DataAnnotations;

public enum StructuredOutputMode
{
    Unspecified,
    NativeJsonSchema,
    PromptedJsonSchema
}

public sealed class SupportModelOptions
{
    [Required]
    public string Provider { get; init; } = "";

    [Required]
    public string Model { get; init; } = "";

    public string? Deployment { get; init; }

    [Required]
    public Uri Endpoint { get; init; } = null!;

    public StructuredOutputMode StructuredOutput { get; init; }
}

An Azure OpenAI configuration can look like this:

{
  "AI": {
    "SupportRouting": {
      "Provider": "AzureOpenAI",
      "Endpoint": "https://example.openai.azure.com/",
      "Model": "your-model-id",
      "Deployment": "support-routing-prod",
      "StructuredOutput": "NativeJsonSchema"
    }
  }
}

The deployment is the Azure OpenAI deployment name used by the SDK. It identifies a model deployment inside the Azure OpenAI resource. Model is the expected model identity the team records with traces, evaluations, and release decisions. They may have the same text, but they own different concerns.

For local Ollama development, only the provider-specific settings change:

{
  "AI": {
    "SupportRouting": {
      "Provider": "Ollama",
      "Endpoint": "http://localhost:11434/",
      "Model": "your-qualified-model",
      "StructuredOutput": "PromptedJsonSchema"
    }
  }
}

StructuredOutput describes a tested capability of the model, backend, and integration. It is not a permanent fact about the provider. Native JSON Schema may work well enough for one combination of model, backend, and integration but not another. Set NativeJsonSchema only after checking the exact combination you will deploy. The probe later in this article checks that the path works, but it cannot prove schema enforcement. Unspecified exists so missing configuration fails instead of silently selecting the stronger mode.

Do not copy the placeholder model names and capability values into production. Qualify the model you intend to run.

Build one provider-neutral client holder

The application needs the chat client and the capability declaration together. A small holder keeps them consistent. The DI container owns that holder and disposes the selected IChatClient when the container shuts down:

using Microsoft.Extensions.AI;

public sealed class SupportModelClient(
    IChatClient chatClient,
    string declaredModelIdentity,
    StructuredOutputMode structuredOutput) : IDisposable
{
    public IChatClient ChatClient { get; } = chatClient;
    public string DeclaredModelIdentity { get; } = declaredModelIdentity;
    public StructuredOutputMode StructuredOutput { get; } = structuredOutput;

    public void Dispose() => ChatClient.Dispose();
}

I would not call this class AiClient. It is the selected client for one application purpose. A document summarizer or an agent may have a different model, capability set, and operating policy.

DeclaredModelIdentity is configuration, not proof of what an Azure deployment currently serves. A deployment can move to another model or version while the application setting becomes stale. Record model metadata returned by the provider as an observed identity when it is available, and compare it with the declared value in telemetry or deployment checks.

Create the provider clients at the edge

The factory is allowed to know the provider SDKs. That is its job.

using Azure.AI.OpenAI;
using Azure.Identity;
using Microsoft.Extensions.AI;
using OllamaSharp;

public static class SupportModelClientFactory
{
    public static SupportModelClient Create(SupportModelOptions options)
    {
        if (options.StructuredOutput is not (
            StructuredOutputMode.NativeJsonSchema or
            StructuredOutputMode.PromptedJsonSchema))
        {
            throw new InvalidOperationException(
                "A supported structured-output mode is required.");
        }

        return options.Provider switch
        {
            "AzureOpenAI" => CreateAzureOpenAI(options),
            "Ollama" => CreateOllama(options),
            _ => throw new InvalidOperationException(
                $"Unsupported AI provider '{options.Provider}'.")
        };
    }

    private static SupportModelClient CreateAzureOpenAI(
        SupportModelOptions options)
    {
        if (string.IsNullOrWhiteSpace(options.Deployment))
        {
            throw new InvalidOperationException(
                "AI:SupportRouting:Deployment is required for Azure OpenAI.");
        }

        var azureClient = new AzureOpenAIClient(
            options.Endpoint,
            new DefaultAzureCredential());

        IChatClient chatClient = azureClient
            .GetChatClient(options.Deployment)
            .AsIChatClient();

        return new SupportModelClient(
            chatClient,
            options.Model,
            options.StructuredOutput);
    }

    private static SupportModelClient CreateOllama(
        SupportModelOptions options)
    {
        IChatClient chatClient = new OllamaApiClient(
            options.Endpoint,
            options.Model);

        return new SupportModelClient(
            chatClient,
            options.Model,
            options.StructuredOutput);
    }
}

Azure authentication belongs in CreateAzureOpenAI. Ollama endpoint and model setup belong in CreateOllama. TicketRouter sees neither SDK.

The switch runs once while the container creates its singleton. It does not appear in every request path, controller, or application service.

Register the options, selected client, and feature service in Program.cs:

using Microsoft.Extensions.Options;

builder.Services
    .AddOptions<SupportModelOptions>()
    .BindConfiguration("AI:SupportRouting")
    .ValidateDataAnnotations()
    .Validate(
        options => options.Provider is "AzureOpenAI" or "Ollama",
        "Provider must be AzureOpenAI or Ollama.")
    .Validate(
        options => options.StructuredOutput is
            StructuredOutputMode.NativeJsonSchema or
            StructuredOutputMode.PromptedJsonSchema,
        "StructuredOutput must be declared explicitly.")
    .Validate(
        options =>
            options.Provider != "AzureOpenAI" ||
            !string.IsNullOrWhiteSpace(options.Deployment),
        "Deployment is required for Azure OpenAI.")
    .ValidateOnStart();

builder.Services.AddSingleton<SupportModelClient>(services =>
{
    SupportModelOptions options = services
        .GetRequiredService<IOptions<SupportModelOptions>>()
        .Value;

    return SupportModelClientFactory.Create(options);
});

builder.Services.AddTransient<TicketRouter>();

The custom validators reject an unsupported provider, a missing capability declaration, and a missing Azure OpenAI deployment during startup. The factory keeps defensive checks for callers that bypass this registration path. A production application should also validate allowed endpoint schemes, the selected authentication mode, and unsupported option combinations.

Implement the feature once

The model-facing DTO uses strings so the application can reject unknown queue values deliberately. Deserializing directly into an enum can make an unsupported value look like a serialization detail instead of an invalid model decision.

internal sealed record TicketRoutingOutput(
    string? Queue,
    bool? RequiresHumanReview,
    string? Reason);

RequiresHumanReview is nullable only at the model boundary. If the model omits the property, deserialization leaves it as null and the application can reject the incomplete decision instead of treating the missing value as false.

The router asks for a typed response, then validates every field that crosses the model boundary:

using Microsoft.Extensions.AI;

public sealed class TicketRouter(SupportModelClient model)
{
    private const string Instructions = """
        Route the support ticket to exactly one queue:
        Billing, TechnicalSupport, or AccountAccess.
        Set requiresHumanReview to true when the ticket is ambiguous.
        Keep the reason to 120 characters or fewer.
        Return JSON only.
        """;

    public async Task<TicketRoutingDecision> RouteAsync(
        string ticket,
        CancellationToken cancellationToken)
    {
        if (string.IsNullOrWhiteSpace(ticket))
        {
            throw new ArgumentException(
                "Ticket text is required.",
                nameof(ticket));
        }

        ChatResponse<TicketRoutingOutput> response =
            await model.ChatClient.GetResponseAsync<TicketRoutingOutput>(
                [
                    new ChatMessage(ChatRole.System, Instructions),
                    new ChatMessage(ChatRole.User, ticket)
                ],
                useJsonSchemaResponseFormat:
                    model.StructuredOutput is
                        StructuredOutputMode.NativeJsonSchema,
                cancellationToken: cancellationToken);

        if (!response.TryGetResult(out TicketRoutingOutput? output) ||
            output is null ||
            output.RequiresHumanReview is null ||
            string.IsNullOrWhiteSpace(output.Reason) ||
            output.Reason.Length > 120)
        {
            throw new InvalidOperationException(
                "The model returned an invalid routing decision.");
        }

        TicketQueue queue = output.Queue switch
        {
            nameof(TicketQueue.Billing) => TicketQueue.Billing,
            nameof(TicketQueue.TechnicalSupport) =>
                TicketQueue.TechnicalSupport,
            nameof(TicketQueue.AccountAccess) => TicketQueue.AccountAccess,
            _ => throw new InvalidOperationException(
                "The model returned an invalid queue.")
        };

        return new TicketRoutingDecision(
            queue,
            output.RequiresHumanReview.Value,
            output.Reason);
    }
}

Both modes use the same DTO and application validation. The generated schema describes the serialized DTO shape. The explicit checks still own domain rules such as required fields, the allowed queue names, and the maximum reason length.

GetResponseAsync<T> derives a JSON Schema from T in both modes. With NativeJsonSchema, the helper sends that schema through the backend’s structured-response mechanism. With PromptedJsonSchema, it requests JSON and appends another user message containing the generated schema. The second mode guides the model through the prompt instead of relying on native schema enforcement. The application applies the same post-deserialization validation either way.

An invalid result should become an explicit application outcome in a complete service layer. I use an exception here to keep the sample focused on provider composition. The production decision may be to return InvalidGeneration, make one bounded repair attempt, or require review. It should not return the unvalidated text as a routing decision.

Treat capabilities as claims that need evidence

A capability flag can become another lie in configuration. Verify it against the exact model, deployment, integration package, and endpoint used by the environment.

A small qualification probe can check the structured-output path:

using Microsoft.Extensions.AI;

internal sealed record CapabilityProbe(string? Value);

public static class SupportModelQualification
{
    public static async Task VerifyAsync(
        SupportModelClient model,
        CancellationToken cancellationToken)
    {
        ChatResponse<CapabilityProbe> response =
            await model.ChatClient.GetResponseAsync<CapabilityProbe>(
                "Return JSON with value set to ready.",
                useJsonSchemaResponseFormat:
                    model.StructuredOutput is
                        StructuredOutputMode.NativeJsonSchema,
                cancellationToken: cancellationToken);

        if (!response.TryGetResult(out CapabilityProbe? result) ||
            result?.Value != "ready")
        {
            throw new InvalidOperationException(
                $"Model '{model.DeclaredModelIdentity}' failed the " +
                "support-routing structured-output check.");
        }
    }
}

Run this as a deployment smoke test or structured-output contract test. I would not make every application startup depend on a paid model call. The test should run whenever the model, deployment, provider integration, or relevant configuration changes.

This probe proves that the request succeeds and the helper can deserialize a compatible result for this example. A model could return {"value":"ready"} without obeying the supplied schema, so the probe does not prove native schema enforcement. It also says nothing about routing quality. Keep a small golden dataset for that question and evaluate every candidate backend against the same cases. Provider independence without comparable output quality is not useful independence.

Keep provider-only behavior in a named adapter

ChatOptions covers common settings. It also exposes AdditionalProperties and RawRepresentationFactory for options understood by a specific provider. Those escape hatches are useful, but the type system cannot turn them into portable behavior.

If TicketRouter creates an OpenAI ChatCompletionOptions through RawRepresentationFactory, that call is OpenAI-specific even though the outer dependency is IChatClient. Put it in an infrastructure adapter with a name that says what it requires, or accept that the feature is tied to that provider.

The same rule applies to hosted file search, provider-managed conversation state, batch APIs, safety configuration, and model-specific reasoning controls. Do not flatten those features into an AiOptions bag shared by the whole application.

Sometimes the honest design has two paths:

portable ticket routing
    -> IChatClient

provider-hosted file search
    -> purpose-specific application interface
        -> provider SDK adapter

That is still a well-contained architecture. Independence is about controlling the dependency, not eliminating every provider reference from the repository.

What changes during a provider switch

With this structure, moving the ticket router to another backend still requires engineering work. The work is contained:

  1. Add the provider package and one factory branch.
  2. Configure endpoint, model identity, deployment identity when applicable, and the qualified capability mode.
  3. Run the structured-output contract check.
  4. Run the routing evaluation dataset and compare quality, latency, and cost.
  5. Review operational differences such as authentication, throttling, telemetry metadata, and content retention.

TicketRouter changes only if the feature contract changes. A new provider that cannot satisfy the existing contract is not a drop-in replacement. It may still be usable behind a deliberately reduced mode, but the application must name and test that behavior.

When this approach is enough

Use this pattern when several backends can satisfy the same bounded operation through IChatClient. Classification, extraction, summarization, and constrained generation are good candidates when their required inputs and outputs fit the shared abstraction.

It also works well when production uses one provider and local development uses another. That setup catches accidental SDK leakage early, even if the production provider never changes.

When provider independence is the wrong goal

Keep the provider SDK visible when the feature exists because of a provider-only capability, or when the abstraction would discard information the application needs. A thin, purpose-specific adapter is better than a generic interface full of string properties and raw objects.

Do not pay a permanent complexity tax for a switch the application will never make. One provider behind a clean composition boundary is already a sensible design. Add alternate implementations and capability negotiation when an environment-specific requirement, resilience plan, procurement constraint, or local-development need justifies them.

Practical takeaway

IChatClient is useful only for behavior it can represent honestly. Keep provider construction at the edge and attach the evidence-based capability mode to the selected client.

Provider-specific code will remain. Keep it easy to find and change, with its purpose stated in the adapter. The application feature should have no reason to care which SDK sits underneath.

Sources