Learning path · Retrieval & Ranking · 49
Indexing Strategies
How and when you chunk, embed, and refresh corpora—batch, incremental, multi-version, and metadata-rich pipelines.
Why it matters
- Stale indexes cause confident wrong answers.
- Incremental updates must respect deletes and permission changes.
- Reindex plans interact with embedding model upgrades.
Key ideas
- Incremental sync
- Versioned embeddings
- ACL propagation
Indexing strategy spans ingestion schedulers, change-data-capture from wikis, tombstoning deleted pages, and blue-green index cuts when embedding models change. Tag chunks with source timestamps so the generator can prefer fresh policy. For regulated domains, log who indexed what and when. Pair indexing SLAs with RAG eval dashboards—recall drops often trace to broken crawlers, not model regressions. Define SLOs for index freshness per corpus tier—marketing pages and pricing tables need tighter bounds than archived research PDFs. Ship only after eval gates pass on representative production failures.
Updated 2026-08-09 · Full learning path