Learning path · Transformers & Attention · 17
Attention Mechanism
Weighted lookup that lets each token focus on the most relevant other tokens when building its representation.
Why it matters
- Core to context usage—models prioritize some tokens over others.
- Explains phenomena like lost-in-the-middle and prompt ordering effects.
- Informs placement of instructions and evidence in prompts.
Key ideas
- Query-key-value
- Softmax weights
- Head specialization
Attention computes, for each token, a distribution over other positions—deciding what to read. Multiple heads can specialize in syntax, coreference, or local patterns. In practice, prompt engineers exploit attention biases: models often weight beginnings and endings of context more heavily than middles. Put critical rules in the system prompt, place the freshest evidence near the question, and never bury mandatory constraints inside a long retrieved dump. When debugging RAG misses, reorder evidence before re-embedding entire corpora—attention bias often beats brute-force index expansion.
Updated 2026-08-09 · Full learning path