Definition
An attention mechanism combines information from a set of representations using weights computed relative to a query. In transformer attention, learned projections turn input representations into queries, keys, and values. Each query is scored against the available keys, and the resulting weights determine how the values are combined.
In self-attention, all three come from the same sequence. In cross-attention, queries come from one source, while keys and values come from another. Multiple heads use separate projections and can produce different weighting patterns in parallel.
Simple example
In “The service rejected the request because it was unauthorized”, the representation for “it” can draw on earlier tokens. One attention head may give “request” more weight than “service” at that position. In an autoregressive model, later layers can use the result when predicting the next token.
The model computes those weights from projected queries and keys, not from a hand-written rule about pronouns.
Why it matters
Attention lets transformer models combine information across a sequence without passing it through a recurrent chain. Its implementation affects the practical cost of long contexts, inference latency, and memory use. During autoregressive generation, a KV cache can reuse keys and values from earlier tokens.
Information in the context is available to the model, but attention does not guarantee the model will use it correctly.
One important nuance
An attention weight is a coefficient used to compute a representation. It does not show what the model “considers important” in a human sense, and a high weight alone cannot explain the final output. Standard dense self-attention also grows more expensive as the sequence gets longer, though other attention patterns and optimized kernels can change the practical cost.