Definition
Time to First Token (TTFT) is the interval from a defined request-start boundary until the first generated output token reaches a defined observation point. A client typically starts the clock when it sends the request and stops when it receives that token. Depending on those boundaries, TTFT can include queueing, request preparation, prompt processing, generation of the first token, and network delivery.
For non-streaming APIs, clients may not observe an individual first token even though the serving system can still measure the equivalent internal milestone.
Simple example
A client sends a streaming request at 10:00:00.000 and receives the first text token at 10:00:00.850. The observed TTFT is 850 milliseconds. The response then continues for another five seconds as later tokens arrive.
Two responses can finish at the same time even if one starts streaming sooner and generates later tokens more slowly.
Why it matters
TTFT captures how quickly a streamed response begins. It is influenced by queue depth, prompt length, batching, model size, hardware, network latency, and whether repeated prompt computation can be reused.
Track TTFT alongside Time per Output Token to separate a slow start from slow ongoing decoding. TTFT alone cannot tell you why the start was slow. To attribute the delay, you need separate timings for queueing, prompt processing, and network delivery. Percentiles are usually more informative than a single average because load and prompt sizes vary.
One important nuance
TTFT is not the same as total response latency or Time per Output Token. A system can produce the first token quickly and then generate the rest slowly. Long prompts often increase prefill work, but context-window size is only a maximum capacity, not the actual prompt length. Use consistent timing boundaries and record relevant request characteristics when comparing TTFT across clients, models, or serving stacks.