Definition
A log probability, often called a logprob, is the natural logarithm of a probability. For an autoregressive language model, a token logprob describes the probability of that token given the preceding context. Probabilities between zero and one become negative log probabilities. A probability of one becomes zero. A value closer to zero means the token was more likely under the distribution being measured.
Simple example
Suppose a next-token distribution assigns one token a probability of 0.8. Its log probability is about -0.223. A token with probability 0.1 gets about -2.303. The first token is more likely, so its log probability is higher.
For a two-token continuation with conditional probabilities of 0.8 and 0.5, add their log probabilities: -0.223 + -0.693 = -0.916. Exponentiating gives a sequence probability of 0.4. With long sequences, adding log probabilities also avoids multiplying tiny numbers until they underflow.
Why it matters
Token logprobs, when an API exposes them, show which tokens in a continuation received low probability under the reported distribution. To score alternative continuations, you need a logprob for every token of each candidate. Sum those values under the same model, context, and scoring setup. For candidates of different lengths, decide whether to rank by total log probability or average log probability per token.
One important nuance
An API might report logprobs from the model’s original distribution or from one modified by temperature or token filtering. Check which distribution it uses before comparing scores. Logits are raw scores before normalization. Logprobs come from probabilities. A value close to zero means a token was likely under that distribution. It does not certify that the resulting statement is true.