Definition
Approximate nearest neighbor search (ANN) seeks vectors close to a query vector using methods designed to reduce the cost of exact search. Depending on the method, it may examine fewer candidates, use approximate distance calculations, or combine both. That can make queries faster as the collection grows, but the results are not guaranteed to be the exact nearest neighbors under the chosen distance measure.
ANN is a way to execute vector search. It does not define the embedding model, distance measure, or number of results to return.
Simple example
Suppose an assistant has one million embedded documentation passages and needs the five closest to a question. A brute-force exact search scores every passage vector. An ANN index visits fewer candidates and returns five close passages. It may skip one passage that would have appeared in the exact top five.
Why it matters
Comparing a query with every vector can become too slow for a large collection or a busy service. ANN gives retrieval systems another way to meet a query latency target. The tradeoff depends on the index and its search settings: searching more candidates generally costs more time but can recover more of the exact neighbors. Building and storing the index also has a cost.
For a RAG application, a missed passage matters when it contains evidence the model needs. Compare ANN results with exact search on representative queries, then check whether the needed passages survive into the retrieved set. Measure latency on the same workload.
One important nuance
Finding the exact nearest vectors is not the same as finding the most useful passages. At a fixed k, recall@k is the fraction of exact top-k neighbors also returned by the ANN search. It does not tell you whether those neighbors answer the user’s question. Evaluate retrieval relevance separately.