Choose rate-limiting policies by endpoint cost

One hundred cached reads and one hundred AI generations do not create the same load. A single global request limit hides that difference.

Define policies around the resource each endpoint consumes. A cheap read may need a time-based limit for fair usage. A model call may need a concurrency limit because each active request holds connections, consumes provider capacity, and can run for several seconds.

This example uses one shared pool for each named policy. Use a partitioned policy when the limit must be enforced per tenant, user, API key, or another bounded identity.

ASP.NET Core lets you attach named policies to individual endpoints:

using Microsoft.AspNetCore.RateLimiting;

builder.Services.AddRateLimiter(options =>
{
    options.RejectionStatusCode =
        StatusCodes.Status429TooManyRequests;

    options.AddFixedWindowLimiter("cheap-read", limiter =>
    {
        limiter.PermitLimit = 100;
        limiter.Window = TimeSpan.FromMinutes(1);
        limiter.QueueLimit = 0;
    });

    options.AddConcurrencyLimiter("ai-generation", limiter =>
    {
        limiter.PermitLimit = 4;
        limiter.QueueLimit = 0;
    });
});

app.UseRateLimiter();

app.MapGet("/products", GetProducts)
   .RequireRateLimiting("cheap-read");

app.MapPost("/documents/{id}/summary", CreateSummary)
   .RequireRateLimiting("ai-generation");

ASP.NET Core defaults rejected requests to 503 Service Unavailable. Set RejectionStatusCode explicitly when clients should receive the 429 Too Many Requests response described here. Use OnRejected instead when the API also needs a response body, logging, or a meaningful Retry-After header.

When the application calls UseRouting() explicitly, call UseRateLimiter() after it. That ordering is required for endpoint-specific policies.

The numbers are only examples. Measure request duration, downstream quotas, memory, CPU, database work, and acceptable queueing before choosing real limits. Put the values in configuration when operators need to tune them without changing endpoint code. For bursty public reads, a token bucket or sliding window policy may smooth traffic better than a fixed window. The example uses a fixed window because the policy is easy to read.

Decide how to partition the limit as well. A shared AI concurrency limit protects the capacity visible to that limiter, but one noisy tenant can consume all permits. A partitioned policy can enforce fairness by tenant or authenticated caller. Do not create unbounded partitions from arbitrary user input.

Remember the deployment boundary. The limiter instances configured here live in the application process. In a scaled-out service, a limit of four concurrent AI requests normally means four per application instance, not four across the whole service. If the limit must protect a shared downstream quota across replicas, enforce it at a gateway or quota service with the required scope, through a queue, at the provider boundary, or with a distributed limiting mechanism.

Queueing is not automatically helpful. QueueLimit = 0 disables it in this example, so queue order has no effect. If you enable a queue, QueueProcessingOrder.OldestFirst prevents newer requests from jumping ahead of older ones. A large rate-limiter queue can still turn rejection into a long timeout while the application holds resources. For expensive AI work, a small queue or immediate 429 Too Many Requests response is often easier for clients to handle. Durable work that must survive the request belongs in a background-job design instead.

Rate limiting reduces pressure. It does not replace authentication, authorization, provider-side quotas, load testing, or protection at the network edge.