Table of Contents

.NET developers can call language models through several provider SDKs. That part is solved.

The harder decision is where provider-specific code ends and application code begins.

Microsoft.Extensions.AI introduces shared abstractions such as IChatClient, common message and response types, and a middleware pipeline that fits normal .NET dependency injection patterns. Provider SDKs still matter, but they can stay near the composition root instead of defining every application service.

Models and providers still behave differently. The abstraction gives the application one stable place to handle the behavior it owns.

The first generation was organized around the SDK

A small AI feature often starts by injecting a provider client directly into the class that needs it.

public sealed class TicketSummarizer(
    AzureOpenAIClient azureClient,
    IConfiguration configuration)
{
    public async Task<string> SummarizeAsync(
        string ticket,
        CancellationToken cancellationToken)
    {
        ChatClient client = azureClient.GetChatClient(
            configuration["AZURE_OPENAI_DEPLOYMENT"]!);

        ClientResult<ChatCompletion> response =
            await client.CompleteChatAsync(
                $"Summarize this support ticket:\n\n{ticket}",
                cancellationToken: cancellationToken);

        return response.Value.Content[0].Text;
    }
}

This code is fine for a spike or a feature that needs provider-specific behavior.

The architectural problem appears when this shape becomes the default across the application:

  • application services know provider client types
  • deployment names and provider options leak into business code
  • streaming, tools, and structured output use different provider-specific shapes
  • testing requires provider SDK objects or network calls
  • logging, telemetry, caching, and retries are reimplemented around individual calls
  • changing a provider becomes an application-wide migration

At that point, the code follows the provider’s model instead of the application’s use case.

IChatClient creates an application boundary

Microsoft.Extensions.AI defines IChatClient as the common chat abstraction. Provider integrations adapt their concrete clients to that interface.

The code samples use the current Microsoft.Extensions.AI and Azure.AI.OpenAI APIs available at the time of writing. Check package versions before copying them into an application.

The provider client still exists, but it is created at the edge of the application:

using Azure.AI.OpenAI;
using Azure.Identity;
using Microsoft.Extensions.AI;

WebApplicationBuilder builder = WebApplication.CreateBuilder(args);

AzureOpenAIClient azureClient = new(
    new Uri(builder.Configuration["AZURE_OPENAI_ENDPOINT"]!),
    new DefaultAzureCredential());

IChatClient providerClient = azureClient
    .GetChatClient(builder.Configuration["AZURE_OPENAI_DEPLOYMENT"]!)
    .AsIChatClient();

builder.Services
    .AddChatClient(providerClient)
    .UseOpenTelemetry(
        sourceName: builder.Environment.ApplicationName);

Application services now depend on the capability they need:

public sealed class TicketSummarizer(IChatClient chatClient)
{
    public async Task<string> SummarizeAsync(
        string ticket,
        CancellationToken cancellationToken)
    {
        ChatResponse response = await chatClient.GetResponseAsync(
            [
                new ChatMessage(
                    ChatRole.System,
                    "Summarize support tickets without inventing facts."),
                new ChatMessage(ChatRole.User, ticket)
            ],
            cancellationToken: cancellationToken);

        return response.Text;
    }
}

The summarizer no longer knows how the provider authenticates, which deployment name is configured, or which middleware surrounds the request.

That split is what matters. The application service owns summarization. The composition root owns the provider connection.

The abstraction is more than provider switching

Provider portability gets most of the attention. I think it is the smaller benefit.

IChatClient gives application code a shared shape for:

  • chat messages and roles
  • complete and streaming responses
  • tools and function calls
  • response metadata and usage when the provider supplies it
  • dependency injection
  • test doubles
  • middleware composition

Teams can build application behavior once around that interface.

A scripted IChatClient can test a summarizer, router, approval flow, or agent without calling a live model. Telemetry middleware can wrap whichever provider is configured, and integration tests or local development can supply another client without changing the application service.

This is ordinary dependency inversion applied to model access.

Middleware turns cross-cutting behavior into composition

The Microsoft.Extensions.AI package provides a chat-client builder that can compose behavior around the underlying client.

builder.Services
    .AddChatClient(providerClient)
    .UseOpenTelemetry(
        sourceName: builder.Environment.ApplicationName)
    .UseFunctionInvocation(
        configure: client =>
            client.MaximumIterationsPerRequest = 5);

The pattern should look familiar to anyone who has worked with HTTP handlers or ASP.NET Core middleware.

The function-invocation layer removes a particularly repetitive piece of provider-specific code. Without it, the application has to inspect each model response for tool calls, invoke the requested functions, add their results to the conversation, and call the model again until it produces a final response or reaches a stop condition. UseFunctionInvocation() handles that loop around the inner client. The iteration limit in the example makes one of those stop conditions explicit.

The host can add shared behavior once instead of repeating it inside every use case:

  • telemetry
  • function invocation
  • caching
  • logging
  • resilience policies
  • request or response inspection
  • custom policy middleware

Order still matters. Middleware is not a bag of attributes. A cache placed outside telemetry observes different behavior from telemetry placed outside the cache. Function invocation introduces additional model calls and tool executions that need their own budgets and traces.

A useful pipeline makes runtime behavior easier to inspect. If the order is difficult to explain, the composition is too clever.

Keep use-case policy outside the pipeline. Authorization, approval rules, data access, argument validation, and side-effect safety still need explicit application boundaries.

Shared interfaces do not erase provider differences

An abstraction can standardize a programming model. It cannot make models equivalent.

Providers and model families still differ in areas such as:

  • tool and function-calling behavior
  • structured-output guarantees
  • multimodal input support
  • token accounting
  • safety systems
  • streaming details
  • response metadata
  • model-specific options
  • rate limits and regional availability

Application code can depend on IChatClient while the composition layer still chooses a concrete provider and exposes required provider-specific capabilities deliberately.

If a feature depends on a provider-only API, represent that dependency honestly. Do not force it through a generic interface until the important behavior disappears.

Keep provider coupling visible and contained. A generic interface that hides provider-only capabilities gives the application a misleading kind of portability.

One application may need more than one client

Registering one global IChatClient is enough for a small application. Larger systems often need clients with different purposes:

  • a fast model for classification
  • a stronger model for complex generation
  • a restricted client for sensitive workflows
  • a local model for development
  • a separate embedding generator for retrieval

Do not hide those differences behind runtime string comparisons throughout the application.

Use explicit typed services, keyed DI registrations, or a small routing abstraction at the composition boundary. Keep stable application names separate from provider deployment names.

IChatClient classificationClient = azureClient
    .GetChatClient(
        builder.Configuration["AZURE_OPENAI_CLASSIFICATION_DEPLOYMENT"]!)
    .AsIChatClient();

builder.Services.AddKeyedChatClient(
    "classification",
    classificationClient);

builder.Services.AddTransient<SupportClassifier>();

public sealed class SupportClassifier(
    [FromKeyedServices("classification")] IChatClient chatClient)
{
    // The application depends on a stable purpose,
    // not a provider deployment name.
}

Register the consuming service itself through DI so the container resolves its keyed dependency.

Use a key such as classification because it describes the application’s purpose. Configuration decides which provider and model serve that purpose.

This helps during local development too. The classification registration can use a local model container during development and an Azure-backed client in the cloud. SupportClassifier stays unchanged because its dependency is the application’s purpose rather than the environment’s provider.

Agents and frameworks sit above this foundation

Not every AI feature needs an agent.

A direct IChatClient call is often enough for summarization, classification, extraction, translation, or another bounded inference task.

Agent and orchestration frameworks become useful when the application needs session state, dynamic tool selection, multi-step workflows, approvals, handoffs, or coordination between agents.

Microsoft Agent Framework uses the Microsoft.Extensions.AI abstractions as a lower layer:

application use case
    -> agent or workflow when needed
    -> Microsoft.Extensions.AI abstractions
    -> provider integration
    -> model endpoint

Start at the lowest layer that solves the problem. Reach for an agent only when the use case needs agent behavior.

A practical migration path

A full platform rewrite is unnecessary. Move one boundary at a time.

1. Find provider clients outside the composition root

Search for application services, handlers, and controllers that construct or depend directly on provider SDK clients.

Do not change provider-specific code that genuinely needs provider-specific APIs. Identify the calls that only need normal chat behavior.

2. Adapt the provider client to IChatClient

Create the provider client and deployment configuration in the host. Adapt it once, then register the resulting IChatClient.

3. Move prompts and use-case policy into application services

Create clients in the composition root. Keep business prompts, authorization rules, and output policy in application services.

4. Add one cross-cutting behavior through the pipeline

I would start with telemetry. It shows which calls already pass through the new boundary before caching, retries, tools, or custom middleware complicate the pipeline.

5. Replace network-heavy tests with a fake client

Test deterministic application behavior with a scripted IChatClient. Keep a smaller set of provider integration tests and separate evaluations for model quality.

This migration produces value even if the provider never changes. The application becomes easier to compose, test, and operate.

When to use the shared abstraction

Use IChatClient when application code needs common chat, streaming, tool, or structured-response capabilities and should participate in normal .NET dependency injection and middleware composition.

It is a strong default for reusable libraries, application services, tests, and cross-cutting telemetry.

When to keep the provider SDK visible

Use the provider SDK directly when the shared abstraction cannot represent a capability cleanly. Examples include provider-specific batch APIs, file management, fine-tuning operations, deployment administration, and specialized request options.

Keep that code close to an adapter or infrastructure service. Do not let one specialized requirement pull provider types through the rest of the application.

Practical takeaway

Provider SDKs connect .NET applications to model platforms. Keep that role at the edge of the application.

Create the provider client at the edge. Let application code depend on IChatClient or another purpose-specific interface. Compose telemetry, tools, caching, and other cross-cutting behavior through explicit middleware. Keep provider differences visible where they matter.

The result looks like the rest of a maintainable .NET application. Services own use cases, while provider setup and shared runtime behavior stay centralized in the host.

Sources