Table of Contents

If you already keep provider SDKs at the edge and model calls behind a use-case service, the next problem is assembling those decisions into a working ASP.NET Core application without inventing an AI platform.

This article builds the smallest foundation I would use for that application: one concrete operation, validated model configuration, an instrumented IChatClient pipeline, and explicit buffered and streaming HTTP contracts.

Retrieval, tools, evaluation, and runtime controls stay out of this first version. The code still needs a sensible place for each one when it becomes a real requirement.

Keep the dependency direction small

The request path in this example is deliberately boring:

HTTP endpoint
    -> SupportReplyService
        -> IChatClient

composition root
    -> configuration
    -> Azure OpenAI client
    -> IChatClient pipeline

The articles on provider abstraction and AI service layers explain these two boundaries in detail. I will treat them as settled here and focus on the host around them.

I care more about those arrows than the number of projects. Everything can live in one ASP.NET Core project at first:

AiAppFoundation/
├── Program.cs
├── AI/
│   └── SupportModelOptions.cs
└── SupportReplies/
    ├── SupportReplyContracts.cs
    └── SupportReplyService.cs

Five class libraries would add more project references and more files to navigate without improving this small dependency graph. Keep the direction clear in code first. Add an assembly when you need to enforce dependencies at compile time, reuse a component independently, or reflect a clear ownership boundary. An independently deployed application will usually have its own assembly, but a class library is not a deployment boundary by itself.

Start with one actual operation

The sample drafts a reply to a customer message. It is intentionally unglamorous. Put the application contract in SupportReplies/SupportReplyContracts.cs:

namespace AiAppFoundation.SupportReplies;

public sealed record DraftReplyRequest(string CustomerMessage);

public sealed record DraftReply(string Text);

The endpoint does not accept a system prompt, arbitrary tools, a model name, or provider options. A caller asks for a support reply. It does not get to reconfigure the feature.

Put the service in SupportReplies/SupportReplyService.cs. It owns the instructions and builds fresh request objects for each operation:

using System.Runtime.CompilerServices;
using Microsoft.Extensions.AI;

namespace AiAppFoundation.SupportReplies;

public sealed class SupportReplyService(IChatClient chatClient)
{
    private const string Instructions = """
        Draft a concise support reply.
        Use only facts present in the customer message.
        If information is missing, ask for it instead of inventing it.
        """;

    public async Task<DraftReply> DraftAsync(
        DraftReplyRequest request,
        CancellationToken cancellationToken)
    {
        ChatResponse response = await chatClient.GetResponseAsync(
            CreateMessages(request),
            CreateOptions(),
            cancellationToken);

        return new DraftReply(response.Text);
    }

    public async IAsyncEnumerable<string> StreamAsync(
        DraftReplyRequest request,
        [EnumeratorCancellation] CancellationToken cancellationToken)
    {
        await foreach (ChatResponseUpdate update in
            chatClient.GetStreamingResponseAsync(
                CreateMessages(request),
                CreateOptions(),
                cancellationToken))
        {
            if (!string.IsNullOrEmpty(update.Text))
            {
                yield return update.Text;
            }
        }
    }

    private static List<ChatMessage> CreateMessages(
        DraftReplyRequest request) =>
    [
        new(ChatRole.System, Instructions),
        new(ChatRole.User, request.CustomerMessage)
    ];

    private static ChatOptions CreateOptions() => new()
    {
        MaxOutputTokens = 500
    };
}

SupportReplyService is registered as a concrete type on purpose. The operation, request, and result already give callers a useful boundary. Add a feature-specific interface when you need to replace the service itself, not simply to hide IChatClient. Designing an AI Service Layer explains why a generic IAiService adds little.

Create the messages and ChatOptions for every call because client implementations may mutate them. Dependency Injection for AI Components covers the broader lifetime and scoped-dependency rules.

Wire configuration, model access, and tracing

The code here targets .NET 10. I tested it with Microsoft.Extensions.AI 10.10.0, Microsoft.Extensions.AI.OpenAI 10.10.0, Azure.AI.OpenAI 2.1.0, Azure.Identity 1.21.0, and OpenTelemetry 1.18.0. These are tested pins, not a claim that they will still be current when you start a new project.

dotnet new web -n AiAppFoundation -f net10.0
cd AiAppFoundation
dotnet add package Microsoft.Extensions.AI --version 10.10.0
dotnet add package Microsoft.Extensions.AI.OpenAI --version 10.10.0
dotnet add package Azure.AI.OpenAI --version 2.1.0
dotnet add package Azure.Identity --version 1.21.0
dotnet add package OpenTelemetry.Extensions.Hosting --version 1.18.0
dotnet add package OpenTelemetry.Instrumentation.AspNetCore --version 1.18.0
dotnet add package OpenTelemetry.Exporter.OpenTelemetryProtocol --version 1.18.0

Put the provider-facing settings in AI/SupportModelOptions.cs:

using System.ComponentModel.DataAnnotations;

namespace AiAppFoundation.AI;

public sealed class SupportModelOptions
{
    public const string SectionName = "AI:SupportReplies";

    [Required]
    public Uri Endpoint { get; init; } = null!;

    [Required]
    public string Deployment { get; init; } = "";
}

Add the corresponding routing information to appsettings.json. It does not contain a credential:

{
  "AI": {
    "SupportReplies": {
      "Endpoint": "https://example.openai.azure.com/",
      "Deployment": "support-replies"
    }
  }
}

Register the options and client in Program.cs:

using AiAppFoundation.AI;
using AiAppFoundation.SupportReplies;
using Azure.AI.OpenAI;
using Azure.Identity;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.Options;
using OpenTelemetry.Trace;

WebApplicationBuilder builder = WebApplication.CreateBuilder(args);
const string AiActivitySourceName = "AiAppFoundation.AI";

builder.Services
    .AddOptions<SupportModelOptions>()
    .BindConfiguration(SupportModelOptions.SectionName)
    .ValidateDataAnnotations()
    .ValidateOnStart();

builder.Services
    .AddOpenTelemetry()
    .WithTracing(tracing => tracing
        .AddSource(AiActivitySourceName)
        .AddAspNetCoreInstrumentation()
        .AddOtlpExporter());

builder.Services
    .AddChatClient(services =>
    {
        SupportModelOptions options = services
            .GetRequiredService<IOptions<SupportModelOptions>>()
            .Value;

        var azureClient = new AzureOpenAIClient(
            options.Endpoint,
            new DefaultAzureCredential());

        return azureClient
            .GetChatClient(options.Deployment)
            .AsIChatClient();
    })
    .UseOpenTelemetry(
        sourceName: AiActivitySourceName,
        configure: telemetry =>
            telemetry.EnableSensitiveData = false);

builder.Services.AddScoped<SupportReplyService>();

WebApplication app = builder.Build();

ValidateOnStart catches missing values before the first user request. It cannot prove that Azure is reachable or that the deployment supports every capability the application might request later. Cover those questions with deployment checks and integration tests.

DefaultAzureCredential keeps secrets out of this configuration. For local development, authenticate with a supported developer credential before starting the application. With Azure CLI:

az login

If your account belongs to more than one Microsoft Entra tenant, sign in to the tenant that contains the Azure OpenAI resource:

az login --tenant <tenant-id>

The signed-in identity also needs access to the resource. For inference, Cognitive Services OpenAI User is one suitable role. Check the operations your application performs and grant only the permissions they require. Microsoft’s keyless authentication guidance covers other local and deployed credential options.

AddChatClient registers the pipeline as a singleton by default. Keep request state, scoped database contexts, and caller-specific tools out of its factory. Bind those dependencies inside the operation that owns them.

UseOpenTelemetry is easy to overread. It instruments the chat-client pipeline, but it does not collect or export those activities by itself. A host without a tracing provider subscribed to that source still receives nothing. In this sample, AddSource subscribes to the MEAI source, ASP.NET Core instrumentation supplies the surrounding request span, and the OTLP exporter forwards both. Without exporter settings, it uses the local default for the selected OTLP protocol. Point it to a remote collector through the normal OpenTelemetry configuration for your environment, such as OTEL_EXPORTER_OTLP_ENDPOINT.

The explicit EnableSensitiveData = false keeps raw prompts, responses, function arguments, and function results out of MEAI telemetry even if OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT is set elsewhere in the environment. Metadata such as token counts can still be recorded.

The generative AI semantic conventions are still experimental, so individual telemetry attributes should not become application contracts. This sample traces the HTTP request and model call. Retrieval, tools, retries, and persistence will need their own spans when they arrive.

Choose the HTTP contract: buffered or streaming

Continue in Program.cs with the route mappings. My default for this operation is a buffered JSON response:

app.MapPost("/support/replies", async (
    DraftReplyRequest request,
    SupportReplyService replies,
    CancellationToken cancellationToken) =>
{
    if (string.IsNullOrWhiteSpace(request.CustomerMessage))
    {
        return Results.BadRequest(new
        {
            error = "customerMessage is required."
        });
    }

    DraftReply result = await replies.DraftAsync(
        request,
        cancellationToken);

    return Results.Ok(result);
});

The complete draft exists before the response starts. The application can validate it, then choose the final status code and JSON response as one unit.

Streaming can make a longer generation feel faster. It also changes the API contract. I would give it a separate endpoint rather than quietly changing the buffered one:

app.MapPost("/support/replies/stream", async (
    DraftReplyRequest request,
    SupportReplyService replies,
    HttpResponse response,
    CancellationToken cancellationToken) =>
{
    if (string.IsNullOrWhiteSpace(request.CustomerMessage))
    {
        response.StatusCode = StatusCodes.Status400BadRequest;
        await response.WriteAsJsonAsync(
            new { error = "customerMessage is required." },
            cancellationToken);
        return;
    }

    response.ContentType = "text/plain; charset=utf-8";

    await foreach (string text in replies.StreamAsync(
        request,
        cancellationToken))
    {
        await response.WriteAsync(text, cancellationToken);
        await response.Body.FlushAsync(cancellationToken);
    }
});

app.Run();

Flushing after every update favors immediate delivery in this small example. Model updates can contain only a few characters, so a production endpoint may buffer tiny deltas and flush them in larger chunks to reduce write and frame overhead. Measure the effect against the latency the UI needs.

Once the response starts, the endpoint cannot replace it with a neat JSON error. The client also needs framing when a stream carries citations, tool progress, usage, or a final outcome. Plain text works here only because text is the entire contract.

Server-Sent Events or another framed protocol will often suit a browser application better. The important decision is earlier: stream because the product benefits from partial output, not because the model API happens to expose it.

Both endpoints pass ASP.NET Core’s request cancellation token through SupportReplyService to the model call. Keep that chain intact. Cancellation is cooperative: it cannot guarantee that provider work or billing stops immediately, and it cannot undo a side effect that has already happened.

Run the application on a fixed local URL:

dotnet run --urls http://localhost:5000

This starts the HTTP server. It does not print a model response in the terminal. Once the application is listening, send a buffered request from another terminal:

curl -X POST http://localhost:5000/support/replies \
  -H "Content-Type: application/json" \
  -d '{"customerMessage":"My order has not arrived yet. What should I do?"}'

The response is JSON with a text property. Its wording depends on the configured model. For the streaming route, tell curl not to buffer the response:

curl --no-buffer -X POST http://localhost:5000/support/replies/stream \
  -H "Content-Type: application/json" \
  -d '{"customerMessage":"My order has not arrived yet."}'

Leave room without building placeholders

I would not create interfaces for retrieval, evaluation, and tools on day one. I would know where each concern belongs before adding it.

CapabilityWhere it attachesBoundary to preserve
RetrievalA capability injected into SupportReplyService before message constructionApplication code chooses allowed data and filters before anything reaches the model.
ToolsPer-operation ChatOptions, with function-invocation middleware in the client pipelineA singleton client must not capture scoped authorization or data services.
Structured outputThe service’s response contract and model callValidate the deserialized result before returning or persisting it.
EvaluationTests and production feedback around the feature contractKeep deterministic application tests separate from model-quality evaluation.
ObservabilityThe request trace and the IChatClient pipelineCorrelate the complete operation, including work outside the provider call.
Runtime controlsThe endpoint, service, and client middlewareCarry one request budget through retries, tools, and model calls.

Treat this as an attachment map, not a backlog. Add a capability when the feature requires it. Retrieval should arrive as an authorized application dependency. Bind tools for the current operation. Start evaluation with a concrete quality question rather than an evaluator package.

What this foundation deliberately does not solve

This sample is an application shape, not a production-ready AI module. It does not yet define:

  • authentication or authorization for the endpoint
  • maximum input size and request rate
  • structured output and response acceptance rules
  • provider timeouts, recovery, or overload behavior
  • content-safety policy
  • tool authorization and side-effect handling
  • retrieval filters and data-access rules
  • evaluation datasets and release gates
  • telemetry redaction and retention
  • deployment health checks or fallback behavior

Those omissions are deliberate. Their design depends on the product, its data, and the consequences of failure. This foundation does not answer those questions. It gives their eventual answers a clear place in the application.

When this shape earns its keep

Use this foundation when a model experiment is becoming an application feature with an HTTP contract, shared configuration, operational requirements, or a likely path toward data and tools. A disposable console experiment can remain smaller and call IChatClient directly.

Before adding retrieval or tools, I would check these points:

  • Provider SDK types appear only at the composition edge.
  • Missing configuration stops the application at startup.
  • The endpoint accepts an application request, not an arbitrary prompt or model name.
  • One use-case service owns the instructions, messages, and result contract.
  • Messages and call options are local to one operation.
  • Buffered and streaming endpoints have explicit response contracts.
  • The caller’s cancellation token reaches the model call.
  • The tracing provider subscribes to the MEAI activity source and has an exporter.
  • MEAI sensitive-data capture is disabled explicitly.

At that point, add the next real requirement where it belongs. The application does not need an AI platform first.

Sources