Definition
Chunking divides a source into smaller units for indexing and retrieval. A chunk might be a paragraph, a few sentences, a section under one heading, or a span of tokens.
Systems often index and retrieve chunks directly, but they may return a larger parent section or document instead. Each chunk should keep a link to its source and useful surrounding structure.
Simple example
Instead of embedding a 40-page operations manual as one item, an ingestion pipeline splits it at headings, then divides unusually long sections. A query about restoring a backup can retrieve the recovery section instead of the whole manual.
Why it matters
Chunks that are too large can mix unrelated topics and consume unnecessary context. Chunks that are too small can lose the conditions needed to interpret a statement. Chunking therefore affects retrieval precision, evidence completeness, citation quality, and context cost.
One important nuance
A maximum token count controls size, but it does not tell you where to split. Keep headings with the text they describe and respect document boundaries. Avoid cutting through tables or code blocks where practical. Overlap can carry context across a split, but too much creates near-duplicate chunks that may yield redundant results. Test boundaries with realistic questions and the facts an answer needs, rather than copying a token count from an example.