Learning path · Embeddings & Representation · 40
Late Chunking
Embed the full document (or large span) first, then derive chunk vectors from internal model states—preserving global context in each piece.
Why it matters
- Mitigates context loss when naive chunking embeds isolated snippets.
- Emerging pattern for long documents with cross-references.
- Heavier compute at index time than naive chunk-and-embed.
Key ideas
- Contextualized chunk vectors
- Long-document encoding
- Index-time cost
Late chunking encodes a large passage once, then pools token-level representations per segment so each chunk vector knows surrounding context. Versus semantic chunking—which splits text first, then embeds each piece independently—late chunking helps when pronouns and definitions span sections. Versus structure-aware chunking, it does not require headings but needs compatible embedding models. Weigh index cost against gains on Recall@K for cross-paragraph questions. Validate late chunking on pronoun-heavy and cross-reference-heavy doc sets where naive splits fail even when semantic chunking looks acceptable.
Updated 2026-08-09 · Full learning path