Learning path · Context Engineering & Caching · 22
Context Window
Maximum tokens the model can attend to in one forward pass—system, history, retrieval, tools, and completion combined.
Why it matters
- Hard cap on evidence, tools, and conversation you can include.
- Larger windows increase cost and latency even when unused.
- Window size does not equal reliable recall—see lost-in-the-middle.
Key ideas
- Token budget
- Input ceiling
- Effective vs nominal context
The context window is a finite workspace shared by every byte of instruction and evidence. Exceed it and providers truncate or reject the request. Product design should budget explicitly: reserve space for output, tool definitions, and safety buffers. Long windows help, but disciplined context engineering—what you omit—is often cheaper than stuffing everything because you can. Expose remaining budget in internal debugging tools so engineers see truncation decisions instead of guessing why evidence vanished. Ship only after eval gates pass on representative production failures.
Updated 2026-08-09 · Full learning path