Learning path · Transformers & Attention · 21
Lost in the Middle
Models often under-use information placed in the middle of long contexts, overweighting beginnings and endings.
Why it matters
- Bad chunk ordering in RAG silently drops the best evidence.
- Long-context windows do not guarantee equal recall across positions.
- Drives reranking, re-ordering, and context-engineering tactics.
Key ideas
- U-shaped attention bias
- Evidence placement
- Re-ranking mitigations
Research and production logs show models miss facts buried mid-prompt even when within the nominal context limit. Mitigate by reranking retrieved chunks, placing the highest-value passages last before the question, summarizing mid-context noise, or using map-reduce patterns over segments. Treat "fits in context window" as necessary, not sufficient—ordering and compression matter as much as retrieval score. Add automated tests that place gold facts in middle positions of synthetic contexts to catch ordering regressions before release. Ship only after eval gates pass on representative production failures.
Updated 2026-08-09 · Full learning path