Embeddings in .NET

An embedding is useful only inside the vector space that produced it.

Generating one in .NET is a short API call. The part worth designing is the contract around that call: which model or encoder produced each vector, how many values it contains, which comparison metric will be used during search, and whether indexed documents and runtime queries use compatible rules.

Microsoft.Extensions.AI gives .NET applications a provider-neutral IEmbeddingGenerator<TInput, TEmbedding> abstraction. It keeps provider SDK types out of the retrieval code, but provider neutrality does not imply vector-space compatibility.

Before the first document is indexed, four choices need to be fixed together: the model, vector shape, comparison metric, and input preparation. The examples below use dense text embeddings for semantic retrieval.

An embedding is a model output, not application data

An embedding model, together with its configuration, maps an input into a vector of numbers with a defined dimensionality.

"stop a worker during shutdown"
-> embedding model
-> [0.018, -0.042, 0.007, ...]

The vector is a compact representation learned by the model. Inputs the model represents as similar should end up near one another under a comparison metric appropriate for that embedding space.

The individual coordinates do not have stable application meanings. Position 37 is not “shutdown behavior,” and a larger value at that position does not make one passage more relevant. The vector works as a whole.

This also means an embedding is lossy. You cannot reconstruct the source passage reliably from the vector, and the vector does not retain the document ID, permissions, version, or source URL. Keep the original text and its metadata. The embedding is an index representation of that source, not a replacement for it.

Similarity scores need context too. Read a cosine score within the embedding model, retrieval setup, and corpus that produced it. Adding candidates does not change the score between two vectors. It can change where a passage ranks and whether it remains in the top results. The score distribution across a corpus also affects whether a fixed threshold is useful. The score is not a universal confidence value, a probability that the passage answers the question, or proof that the passage is correct.

Embeddings help rank candidates. They do not own truth or authorization.

Documents and queries must enter the same vector space

A basic semantic retrieval path has two sides.

During ingestion:

document chunk
-> embedding generator
-> document vector
-> vector index

During a query:

user question
-> same embedding contract
-> query vector
-> nearest-neighbor search

The search works because both vectors are coordinates in the same space. Their text does not need to contain the same words. The configured model or encoders should place semantically related inputs close enough for the search step to find them.

“Same embedding contract” is more precise than “same number of floats.” Two models can return vectors with equal dimensions and still use unrelated coordinate systems. Vectors from independently chosen models generally cannot be compared meaningfully. They are compatible only when the models or encoders are explicitly designed to produce representations in the same retrieval space.

This is the first boundary to keep clear after deciding that a feature really needs retrieval. Why Retrieval Exists covers that earlier decision. Once vector search is part of the retrieval path, the embedding contract becomes part of the index schema.

Generate an embedding through IEmbeddingGenerator

For an OpenAI-backed example, install the Microsoft.Extensions.AI.OpenAI package:

dotnet add package Microsoft.Extensions.AI.OpenAI --version 10.9.0

I compiled the sample below with .NET 10 and Microsoft.Extensions.AI.OpenAI 10.9.0 from the public NuGet feed. It uses the IEmbeddingGenerator API. Check package versions before copying it into an application.

using Microsoft.Extensions.AI;
using OpenAI;

string apiKey = Environment.GetEnvironmentVariable("OPENAI_API_KEY")
    ?? throw new InvalidOperationException(
        "Set OPENAI_API_KEY before running the sample.");

IEmbeddingGenerator<string, Embedding<float>> generator =
    new OpenAIClient(apiKey)
        .GetEmbeddingClient("text-embedding-3-small")
        .AsIEmbeddingGenerator();

ReadOnlyMemory<float> queryVector = await generator.GenerateVectorAsync(
    "How do I stop a background service cleanly?",
    cancellationToken: CancellationToken.None);

Console.WriteLine($"Generated {queryVector.Length} values.");

GenerateVectorAsync is a convenience method for one input. It returns the vector as ReadOnlyMemory<float>, which can be passed to a vector-store client or copied into a persistence type when the client requires an array.

The provider-specific client is created at the composition boundary. Application code can depend on IEmbeddingGenerator<string, Embedding<float>> instead of OpenAIClient. Another provider can supply the same interface, but its vectors still have to belong to the index’s embedding space. Changing to an incompatible model, model revision, or embedding configuration requires reindexing.

Keep the API key in the environment, a development secret store, or a managed secret service. Do not put it in source code or configuration committed to the repository.

Generate document embeddings in batches

Ingestion rarely embeds one passage at a time. Send a supported batch so the provider can process several inputs in one request.

string[] passages =
[
    "Use CancellationToken to stop long-running work cooperatively.",
    "Register hosted services with AddHostedService.",
    "A bounded channel can apply backpressure to producers."
];

(string Value, Embedding<float> Embedding)[] embeddedPassages =
    await generator.GenerateAndZipAsync(
        passages,
        cancellationToken: cancellationToken);

foreach ((string passage, Embedding<float> embedding) in
    embeddedPassages)
{
    await index.UpsertAsync(
        text: passage,
        vector: embedding.Vector,
        cancellationToken);
}

GenerateAndZipAsync keeps each input beside its returned embedding. That pairing matters once the pipeline also carries stable chunk IDs, document versions, and payload metadata.

Batch size is an operational setting, not a constant copied from this example. Providers impose different input-count and token limits. Start with a conservative size, pass cancellation through the complete ingestion path, and observe latency and throttling before increasing concurrency.

For a real pipeline, the value sent to UpsertAsync should also include a stable point ID and the metadata needed for filtering and traceability. Keep the collection schema and indexing details in the vector-store layer.

Treat the embedding profile as an index contract

The C# type only says that the vector contains float values. It does not tell the compiler which embedding space, dimensions, or comparison rules belong to those values.

Make that missing context explicit as an embedding profile.

public sealed record EmbeddingProfile(
    string Id,
    string Provider,
    string ModelIdentity,
    int Dimensions,
    string ComparisonMetric,
    string InputMode,
    string PreprocessingVersion);

public static class SupportEmbeddings
{
    public static readonly EmbeddingProfile V1 = new(
        Id: "support-text-v1",
        Provider: "OpenAI",
        ModelIdentity: "text-embedding-3-small",
        Dimensions: 1536,
        ComparisonMetric: "Cosine",
        InputMode: "default",
        PreprocessingVersion: "plain-text-v1");
}

The model and 1,536 dimensions match the default output used above. Cosine is the comparison metric selected for the index and is consistent with OpenAI’s recommendation. If you request a different output size or select another model, the profile must change.

The profile does not need to be this record. It can live in typed options plus deployment metadata, as long as the application can answer which contract produced an index.

ModelIdentity should identify the model and, when the provider exposes one, the specific revision or snapshot. A deployment identifier can be tracked separately when it is operationally useful, but it does not define compatibility by itself.

Provider is provenance, not a compatibility rule. Keep it because operators may need to trace how an index was produced. Changing that value alone does not prove that the vector space changed.

InputMode and PreprocessingVersion sound similar, but they own different decisions. InputMode records model-specific behavior such as query and document roles or prefixes. The OpenAI example uses default because it does not set a model-specific mode. PreprocessingVersion identifies how the application transforms the source text.

I would still give the complete profile a stable ID. A model name on its own leaves too much unsaid.

Keep compatibility decisions and useful provenance together:

Qdrant, for example, requires vectors stored under one vector configuration to share a dimensionality and comparison metric. Named vectors can define separate configurations inside the same collection. Its collection documentation also notes that the metric should follow how the encoder was trained.

For the OpenAI model in this example, metric choice needs one more qualification. OpenAI recommends cosine similarity and states that its embeddings are normalized to length 1. Cosine similarity can therefore be calculated with a dot product. Cosine and Euclidean comparison also produce the same ranking for these embeddings, although their score values have different semantics.

The database can reject a vector with the wrong length. It cannot detect that a 1,536-value query came from the wrong 1,536-dimensional model. That failure is more dangerous because the request succeeds.

Validate the contract before indexing

Configuration drift should fail early, not after a retrieval-quality incident.

One small startup or integration check can verify the output shape:

public static async Task VerifyProfileAsync(
    IEmbeddingGenerator<string, Embedding<float>> generator,
    EmbeddingProfile profile,
    CancellationToken cancellationToken)
{
    ReadOnlyMemory<float> probe = await generator.GenerateVectorAsync(
        "embedding-profile-check",
        cancellationToken: cancellationToken);

    if (probe.Length != profile.Dimensions)
    {
        throw new InvalidOperationException(
            $"Embedding profile '{profile.Id}' expects " +
            $"{profile.Dimensions} dimensions but the configured " +
            $"generator returned {probe.Length}.");
    }
}

This check catches a dimensional mismatch. It does not prove that the configured model is the intended one because two models may return the same size. Verify the model identity and embedding configuration as well. Track the deployment separately when it matters for operations.

Also check the vector-store schema against the same profile before starting ingestion. The generator, collection, and query path should not each carry their own independent copy of the expected dimension.

Treat an incompatible model change as a data migration

An incompatible embedding model uses a different coordinate system from the existing index.

If only the query-side generator changes, its vectors cannot be compared meaningfully with the existing document vectors. If only new documents use the new profile, the collection ends up split across two spaces. Neither failure has to produce an exception.

For this kind of change, I create a new embedding profile and re-embed the corpus. The cost is visible. Mixing vector spaces can fail quietly, which is worse.

A migration usually needs these stages:

1. Create storage for the new profile with its dimensions and comparison metric.
2. Backfill existing source records through the new generator.
3. Write new or changed records to both profiles while the backfill runs, if the application must stay current.
4. Evaluate the new index with known queries and expected sources.
5. Switch reads to the new profile.
6. Retire the old vectors after a rollback window.

Qdrant supports migrations through a new collection and alias swap. Qdrant 1.18 and later can also add a named vector for a second model, which allows backfilling inside the existing collection. Its embedding-model migration guide shows the dual-write sequence.

Do not compare raw similarity scores from the old and new profiles as if they shared a scale. Evaluate whether each profile retrieves the expected sources at the rank positions your application uses.

Test retrieval, not attractive coordinates

Printing a few vector values proves that the API returned numbers. A two-dimensional chart can make a demo easier to understand. Neither one tells you whether the model works for your corpus.

Build a small evaluation set before committing to a profile:

• representative user questions
• the chunks or documents that should be retrieved
• difficult wording differences and domain terms
• cases where lexical search should beat semantic search
• languages the application must support

Then measure whether the expected sources appear in the top results. Compare candidate models, preprocessing rules, and hybrid-search choices against the same set.

Model descriptions and public benchmarks can narrow the options. Your retrieval questions decide whether the profile is good enough for the application.

When embeddings fit

Use embeddings when the application needs semantic ranking across a candidate corpus and exact wording is unreliable. Typical examples include finding a runbook from a problem description, locating related support cases, or selecting documentation passages for RAG.

Do not use an embedding when the request already contains an exact identifier, when structured filters or SQL express the question directly, or when application code owns the decision. Keep permissions and tenant boundaries in trusted filters. A close vector is not an authorization result.

Embeddings are also a poor substitute for source metadata. Store the document ID, version, language, permissions, and original text beside the vector or in a system that the indexed point references.

Keep the contract visible

The minimal .NET call is simple:

ReadOnlyMemory<float> vector =
    await generator.GenerateVectorAsync(text, cancellationToken: token);

The production code needs to keep more than the returned values:

• a provider-neutral IEmbeddingGenerator at the application boundary
• an explicit embedding profile for the embedding space and dimensions
• a comparison metric chosen for that embedding space
• reproducible input preparation
• a migration path that re-embeds existing data
• retrieval tests with expected sources

If those decisions are visible, the next step is mechanical: create a vector-store collection that matches the profile, then index chunks with stable IDs and useful payload data.

Further reading

• Microsoft Learn: Use the IEmbeddingGenerator interface
• Microsoft Learn: GenerateVectorAsync
• Microsoft Learn: GenerateAndZipAsync
• NuGet: Microsoft.Extensions.AI.OpenAI 10.9.0
• OpenAI: Embeddings guide
• Qdrant: Collections
• Qdrant: Migrate to a new embedding model